Industrialization: How to write specific NVS value during the flash process

Question

I wrote a firmware to run our ESP32-based custom PCB. The firmware holds a unique S/N (serial number) in the NVS thru Preferences API which is set thru the bluetooth app I wrote.

But now I have to produce tens of PCB and it takes time to connect and set the S/N thru the app.

The current process is a 2-step process which I want to streamline:

1. flash the generic firmware
2. set the unique S/N

I am wondering if I could write a script that could do both steps, providing the S/N as the script argument.

I could take advantage of Espressif esp tool write_flash for example.

How could I do that?

Imprinting

Answer 1

You can pre-generate the NVS data partition and flash it together with firmware. ESP IDF provides the NVS Partition Generator Utility specifically for for that purpose. Here's an overview of the process.

1. First you create a CSV file (let's name it mfgdata.csv) with your pre-generated data, e.g. serial number, product ID, keypair, whatever. Assume the NVS namespace is "mfgdata_ns".

key,type,encoding,value 
mfgdata_ns,namespace,,
serial,data,string,"ABC1234"
private_key,file,string,/path/key.pri

2. Then you generate the binary partition with NVS data on it (named mfgdata.bin) from this CSV file. Assume the NVS partition starts at address 0x3000.

$IDF_PATH/components/nvs_flash/nvs_partition_generator/nvs_partition_gen.py generate mfgdata.csv mfgdata.bin 0x3000

3. Finally you flash the NVS partition together with your firmware. Here's a sample flashing command for the NVS partition alone (assuming the partition is named "mfgdata_part" in your partition table).

$ $IDF_PATH/components/partition_table/parttool.py -p /dev/ttyUSB0 -b 921600 write_partition --partition-name=mfgdata_part --input=mfgdata.bin

4. Firmware starts up, loads the NVS namespace and finds all values that you specified in mfgdata.csv.

You have to be careful when creating the CSV file. Some 2 years ago they were using the python CSV module to parse this file and it worked perfectly, as expected. Then some not very bright person decided to ditch the python module and replace it by splitting each line on comma character, silently ignoring all parsing problems. I don't know if they've unf*cked it yet.

Getup

Thanks very much @Getup (Loved your last notice and remark) – Novah

@Getup is the parsing problem throw error like so: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte. I reproduce your steps to write value on NVS partition and if I don't use --partition-table-offset 0x9000 I am able to write in NVS but I can't read values even if there are indeed written in NVS partition. – Stevenson

@Stevenson from this description it's not possible to understand what you're trying to do or what goes wrong. Please post a new question with a minimal reproducible example as usual. – Getup