EEPROM parameters structure for small embedded device

Question

The main issue I am addressing in the small embedded device redesign (PID controller) is the device parameters storage. The old solution I partially present here was space efficient, but clumsy to maintain when new parameters were added. It was based on the device parameter ID that had to match th EEPROM address like in an example given below:

// EEPROM variable addresses

#define EE_CRC                       0          // EEPROM CRC-16 value

#define EE_PROCESS_BIAS              1          // FLOAT, -100.00 - 100.00 U
#define EE_SETPOINT_VALUE            3          // FLOAT, -9999 - 9999.9
#define EE_SETPOINT_BIAS             5          // CHAR, -100 - 100 U
#define EE_PID_USED                  6          // BYTE, 1 - 3
#define EE_OUTPUT_ACTION             7          // LIST, DIRE/OBRNU
#define EE_OUTPUT_TYPE               8          // LIST, GRIJA/MOTOR

#define EE_PROCESS_BIAS2             9          // FLOAT, -100.00 - 100.00 U
#define EE_SETPOINT_VALUE2          11          // FLOAT, -9999 - 9999.9
#define EE_SETPOINT_BIAS2           13          // CHAR, -100 - 100 U
#define EE_PID_USED2                14          // BYTE, 1 - 3
#define EE_OUTPUT_ACTION2           15          // LIST, DIRE/OBRNU
#define EE_OUTPUT_TYPE2             16          // LIST, GRIJA/MOTOR

#define EE_LINOUT_CALIB_ZERO        17          // FLOAT, -100.0 - 100.0
#define EE_LINOUT_CALIB_GAIN        19          // FLOAT, -2.0 - 2.0

Every address was hardcoded, and the next address was defined depending on the previous data size (note the uneven spacing between addresses). It was efficient as no EEPROM data storage was wasted, but diffcult to expand without introducing bugs.

In other parts of the code (i.e. HMI menus, data storage...) the code would use parameter list matching the addresses just given, something like the following:

// Parameter identification, NEVER USE 0 (zero) as ID since it's NULL
// Sequence is not important, but MUST be same as in setparam structure

#define ID_ENTER_PASSWORD_OPER             1 
#define ID_ENTER_PASSWORD_PROGRAM          2 
#define ID_ENTER_PASSWORD_CONFIG           3 
#define ID_ENTER_PASSWORD_CALIB            4 
#define ID_ENTER_PASSWORD_TEST             5 
#define ID_ENTER_PASSWORD_TREGU            6 

#define ID_PROCESS_BIAS                    7
#define ID_SETPOINT_VALUE                  8
#define ID_SETPOINT_BIAS                   9
#define ID_PID_USED                       10 
#define ID_OUTPUT_ACTION                  11
#define ID_OUTPUT_TYPE                    12

#define ID_PROCESS_BIAS2                  13

...                        

Then in code using those parameters, for example in the user menu structrues given below, I have built items using my own PARAM type (structure):

struct param {                      // Parametar decription
   WORD   ParamID;                    // Unique parameter ID, never use zero value
   BYTE   ParamType;                  // Parametar type
   char   Lower[EDITSIZE];            // Lowest value string
   char   Upper[EDITSIZE];            // Highest value string
   char   Default[EDITSIZE];          // Default value string
   BYTE   ParamAddr;                  // Parametar address (in it's media)
};                                  

typedef struct param PARAM;

Now the list of parameters is built as array of structures:

PARAM code setparam[] = {
  {NULL, NULL, NULL, NULL, NULL, NULL},                   // ID 0 doesn't exist

  {ID_ENTER_PASSWORD_OPER, T_PASS, "0", "9999", "0", NULL},
  {ID_ENTER_PASSWORD_PROGRAM, T_PASS, "0", "9999", "0", NULL},
  {ID_ENTER_PASSWORD_CONFIG, T_PASS, "0", "9999", "0", NULL},
  {ID_ENTER_PASSWORD_CALIB, T_PASS, "0", "9999", "0", NULL},
  {ID_ENTER_PASSWORD_TEST, T_PASS, "0", "9999", "0", NULL},
  {ID_ENTER_PASSWORD_TREGU, T_PASS, "0", "9999", "0", NULL},  

  {ID_PROCESS_BIAS, T_FLOAT, "-100.0", "100.0", "0", EE_PROCESS_BIAS},
  {ID_SETPOINT_VALUE, T_FLOAT, "-999", "9999", "0.0", EE_SETPOINT_VALUE},
  {ID_SETPOINT_BIAS, T_CHAR, "-100", "100", "0", EE_SETPOINT_BIAS},
  {ID_PID_USED, T_BYTE, "1", "3", "1", EE_PID_USED},
  {ID_OUTPUT_ACTION, T_LIST, "0", "1", "dIrE", EE_OUTPUT_ACTION},
  {ID_OUTPUT_TYPE, T_LIST, "0", "1", "GrIJA", EE_OUTPUT_TYPE},

  {ID_PROCESS_BIAS2, T_FLOAT, "-100.0", "100.0", "0", EE_PROCESS_BIAS2},

...

In essence, every parameter has it's unique ID, and this ID had to match the hardcoded EEPROM address. Since the parameters were not fixed in size, I could not use the parameter ID itself as an EEPROM (or other media) address. The EEPROM organization in the example above was 16-bit word, but it does not matter in principle (more space is wasted for chars so I would prefer 8-bit organization in the future anyway)

The question:

Is there a more elegant way to do this? Some hash table, well known pattern, standard solution for similar problems? EEPROMS are much larger in size now, and I would not mind using the fixed parameter size (wasting 32 bits for boolean parameter) in exchange for more elegant solution. It looks like with fixed size parameters, I could use the parameter ID as the address. Is there an obvious downside in this method that I do not see?

I am now using the distributed HW (HMI, I/O and main controller are separated), and I would like to use the structure in which all devices know about this parameter structure, so that for example remote I/O knows how to scale input values, and the HMI knows how to display and format data, all based only on the parameter ID. I other words, I need single place where all parameters would be defined.

I did my Google research, very little could be found for small devices not icluding some data bases. I was even thinking about some XML definition which would generate some C code for my data structures, but maybe there was some elegant solution more appropriate for small devices (up to 512 K Flash, 32 K RAM)?

Undermanned

Answer 1

I’m not sure whether this is actually better than what you have, but here is an idea. For easier maintenance, consider encapsulating the knowledge of the EEPROM addresses into an “eeprom” object. Right now you have a parameter object and each instance knows where its data is stored in physical EEPROM. Perhaps it would be easier to maintain if the parameter object had no knowledge of the EEPROM. And instead a separate eeprom object was responsible for interfacing between the physical EEPROM and parameter object instances.

Also, consider adding a version number for the EEPROM data to the data saved in EEPROM. If the device firmware is updated and the format of the EEPROM data changes, then this version number allows the new firmware to recognize and convert the old version of the EEPROM data.

Dandelion

Thanks, it is good suggestion which removes dependency on specific storage organization. I use FreeRTOS, so what you refer to as an 'eeprom' object would be in my case some task receiving messages with param ID and command (store, get, check, backup...) and handling storage to EEPROM(s) or any other media. I could have all parameter structures with constant data (default, min, max...) stored in this immutable object, and I should make sure that any task requiring specifics about internal data organization should ask this 'parameter' object (or task) whatever it needs to know. – Undermanned

And yes, having a version for data identified is a must, otherwise all kind of bad things could happen if some old EEPROM was used on a new board, or during the FW update on the same device. The old version was just overwriting data with default values from the 'default' field for each parameter, safe but time consuming when we wanted to preserve the customer configuration. The device was EPROM based so no firmware update during the operation was possible anyway, something that is considered standard today. – Undermanned

Answer 2

If you are not worried about compatibility across changes or processors, you could simply copy the struct between RAM and EEPROM, and only ever access individual members of the RAM copy.

You could also relatively easily create a tool which would compile a list of defines from the struct and known packing rules of your compiler, if you do want to do explicit access to individual members directly in the EEPROM.

Reseat

Thanks, that was exactly what my main code was doing. I never actually used parameters directly from the EEPROM, except when something was changed. The start of the main control loop (every 100 ms) would check the CRC of the RAM block where parameters were copied from the EEPROM, and if there was a change compared to a previous CRC, the whole parameter block would get refreshed. – Undermanned

You second suggestion is also worth investigating. I hate to depend on the compiler specifics, but I could access the storage media (EEPROM, NVRAM, SD Card) directly only within the RTOS task (let's call it taskDeviceConfig) responsible for non volatile storage. When message with request for specific parameter ID is received, this task (and this task only) would perform whatever magic was needed to extract the single value and send it to the receiver's queue. – Undermanned

Generally, all interacting components running on a system will have a common base idea of struct packing, as interchanging structs is common in many OS service functions. So I don't think you need to limit yourself to a single accessor for that reason. Also you can with moderate care compute the offset of an element into a struct at runtime with pointer arithmetic. – Reseat

True, and pointer arithmetic is so much easier if the element size is not variable. Also, it is easier to reclaim some unused space, or reuse space previously taken by older parameters. I think I am going to use fixed size parameters for a single line of product. – Undermanned

Answer 3

Here is wha I would do.

I would create a typedef of a structure with the variables you whish to have in the EEPROM.

Using your example it would look something like this:

typedef struct eeprom_st
{
    float process_biass;
    float setpoint_value;
    char setpoint_bias;
    ....
} eeprom_st_t;

Than I would create an offset define to mark where the structure is to be stored in the EEPROM.

And I would add a pointer to that type to use it as a dummy object:

#define EEPROM_OFFSET 0
eeprom_st_t *dummy;

Than I would use offsetof to get the offset of the specific variable I need like this:

eeprom_write( my_setpoint_bias, EEPROM_OFFSET + offsetof(eeprom_st_t,setpoint_bias),
sizeoff(dummy->setpoint_bias));

To make it more elegant I would turn the eeprom write routine into a macro as well.

Targum