Safely storing and accessing EEPROM

Question

I've recently established the need to store infrequently-updated configuration variables in the EEPROM of a microcontroller. Adding state to the program immediately forces one to worry about

• detection of uninitialized data in EEPROM (i.e. first boot),
• converting or invalidating data from old firmware versions, and
• addressing of multiple structures, each of which which may grow in firmware updates.

Extensive Googling has only turned up one article that addresses keeping your EEPROM data valid through firmware updates. Has anyone used the approach discussed in that article? Is there a better alternative approach?

Zacarias

Answer 1

Personally, I prefer a "tagged table" format.

In this format, your data is split up into a series of "tables". Each table has a header that follows a predictable format and a body that can change as you need it to.

Here's an example of what one of the tables would look like:

Byte 0: Table Length   (in 16-bit words)
Byte 1: Table ID       (used by firmware to determine what this data is)
Byte 2: Format Version (incremented every time the format of this table changes)
Byte 3: Checksum       (simple sum-to-zero checksum)
Byte 4: Start of body
...
Byte N: End of body

I wasn't storing a lot of data, so I used a single byte for each field in the header. You can use whatever size you need, so long as you never change it. The data tables are written one after another into the EEPROM.

When your firmware needs to read the data out of the EEPROM, it starts reading at the first table. If the firmware recognizes the table ID and supports the listed table version, it loads the data out of the body of the table (after validating the checksum, of course). If the ID, version, or checksum don't check out, the table is simply skipped. The length field is used to locate the next table in the chain. When firmware sees a table with a length of zero, it knows that it has reached the end of the data and that there are no more tables to process.

I find this format flexible (I can add any type of data into the body of a table) and robust (keep the header format constant and the data tables will be both forward- and backwards-compatible).

There are a couple of caveats, though they are not too burdensome. First, you need to ensure that your firmware can handle the case where important data either isn't in the table or is using an unsupported format version. You will also need to initialize the first byte of the EEPROM storage area to zero (so that on the first boot, you don't start loading in garbage thinking that it's data). Since each table knows its length it is possible to expand or shrink a table; however, you have to move the rest of the table storage area around in order to ensure that there are no "holes" (if the entire chain of tables can't fit in your device's memory, then this process can be annoying). Personally, I don't find any of these to be that big of a problem, and it is well worth the trouble I save over using some other methods of data storage.

Buford

This sounds like a great solution to deal with changes in table size between firmware versions. Do you have any advice for making it tolerant to power-loss mid-write besides duplicating the data? – Zacarias

Write the entire table with an invalid checksum, then (in a separate transaction) go back and write the correct checksum. If there's a problem mid-write, your checksum will be wrong when you read in the data. – Buford

That is an elegant solution that is trivial to implement: thanks for following up. – Zacarias

Answer 2

Nigel Jones has covered some of the basics in your reference. There are plenty of alternatives.

One alternative, of you have lots of room, is storing key-value pairs instead of structures. Then you can update one value (by appending it) without erasing everything. This is most useful in devices that have a limited number of erase cycles. Your read routine will need to scan from the beginning, updating values each time the key is encountered. Of course your update routine will need to have a "garbage collector" that kicks in when the memory is full.

To handle device errors and power-downs in the middle of updates, we usually store multiple copies of the data. The simplest approach is to pingpong between to halves of the device using sequence number to determine which is newer. A CRC on each section is used to validate it. This also addresses the uninitialized data issue.

For the key-value version you'd need to append the new CRC after each write.

Jabalpur

I really like the sound of this approach in theory, but unfortunately don't have enough EEPROM available to make it feasible: only 1 kB on an ATmega328p. Thanks for the description! – Zacarias