Key-Value pairs in FLASH Memory: file-system-less minINI

Many embedded systems application need to store some kind of data in a persistent way: calibration values, settings or log information. For a smaller amount of data, using an external memory or file system is an overkill. In many system I’m using minINI to store key-value pars in in a ‘ini-file’ way, but it requires the use of a file system of some kind. minINI is great and efficient, and makes getting and storing data really easy. But for simple cases, a single FLASH memory page or sector is just all what I need. Instead managing that page directly, why not using minINI without a file system?

For example I use the above board to drive up to 4 different stepper motors with hall sensors. I need to store calibration offsets and motor information, and the information can be different for each board. The boards are connected by a RS-485 bus, so each board has a unique address. For this application I need to store the key-value pairs (e.g. address=0x10). For this, I’m using minINI which stores the data in a page of the internal FLASH memory, so no file system is needed.

Outline

Because minINI works with files, it requires some kind of file system. For smaller system or for smaller data amount this adds lots of complexity with increased code size and data needs. In this article I present an approach for a file-less and file-system-less adaption of the minINI library, which only requires a small and configurable RAM buffer plus a single FLASH page or sector. That way a very versatile and user-friendly key-value storage can be used for small embedded systems without the need for a file system.

The implementation is available on GitHub.

.ini Files

.ini files allow me to group data into ‘sections’ and ‘key-value’ pairs. Below is an example:

[motor0]
name=split-flap 0
offset=30
x=0

[motor1]
offset=32

Using minINI I can read/write such values. It features a ‘read-only’ mode in which I can only read data.

minINI Glue

The minINI architecture includes a ‘glue’ interface. That way it can be easily adapted to different file systems. Below a ‘glue’ implementation for the FatFS:

#define INI_FILETYPE    FIL
#define ini_openread(filename,file)   (f_open((file), (filename), FA_READ+FA_OPEN_EXISTING) == FR_OK)
#define ini_openwrite(filename,file)  (f_open((file), (filename), FA_WRITE+FA_CREATE_ALWAYS) == FR_OK)
#define ini_close(file)               (f_close(file) == FR_OK)
#define ini_read(buffer,size,file)    f_gets((buffer), (size),(file))
#define ini_write(buffer,file)        f_puts((buffer), (file))
#define ini_remove(filename)          (f_unlink(filename) == FR_OK)

#define INI_FILEPOS                   unsigned long//DWORD
#define ini_tell(file,pos)            (*(pos) = f_tell((file)))
#define ini_seek(file,pos)            (f_lseek((file), *(pos)) == FR_OK)

FLASH Glue

The idea was to add a new ‘glue’ which does not require a file system. Instead, it directly reads and writes FLASH memory and kind of mimics a file system.

This comes with a limitation: only a single ‘ini file’ is supported. That limitation could be easily extended to support multiple ‘files’, but I really did not see the need, at least not in my applications.

Configuration

Configuration is with using configuration macros (see Different Ways of Software Configuration). Below an example configuration:

That way the flash memory can be configured. The data size macro is used to limit the data size to reduce the amount of memory buffer which is used for writing the flash memory.

The used FLASH memory needs to be reduced/excluded from the amount of FLASH available for the linker, otherwise it might be used by the application too. Below is how the memory can be reduced from the memory map, using the NXP MCUXpresso IDE:

Command line

The implementation includes a command line/shell interface.

Below the status of a read-only configuration:

The ‘McuMinINI’ shows the ‘core’ information of minINI, while the ‘ini’ group is for the flash memory implementation of it.

With the command line interface, the data can be inspected or modified:

Usage

Using key-value pairs with minINI is really easy: I can query integer, boolean, floating point or string values. Below an example getting the stored address of the device:

#define NVMC_MININI_FILE_NAME         "settings.ini" /* 'file' name used */

/* RS-485 bus settings */
#define NVMC_MININI_SECTION_RS485     "rs485"
#define NVMC_MININI_KEY_RS485_ADDR    "addr" /* long value: RS-485 address */
...
*addr = McuMinINI_ini_getl(NVMC_MININI_SECTION_RS485, NVMC_MININI_KEY_RS485_ADDR, 0x1, NVMC_MININI_FILE_NAME);

What I really like with minINI is that I can provide a default value (0x1 in above example): if the section/key is not present, it returns the default value I have specified: that way the default values do not need any space in the configuration area.

Storing a value is like in the example below:

McuMinINI_ini_putl(NVMC_MININI_SECTION_RS485, NVMC_MININI_KEY_RS485_ADDR, addr, NVMC_MININI_FILE_NAME);

Memory requirements

Of course every feature comes with some costs. But in this case, I think it is really minimal.

For storing the data in FLASH, it needs at least one sector or flash page. The size of this depends on the system used and is for example 1 kByte (LPC845) or 2 KByte (Kinetis K22), depending on the microcontroller. The size is configured with McuMinINI_CONFIG_FLASH_NVM_BLOCK_SIZE.

For write access it needs a RAM buffer to backup and store the data from FLASH. The amount is configured McuMinINI_CONFIG_FLASH_NVM_MAX_DATA_SIZE. The minimum amount is 64 bytes. Depending on the amount of data to be stored, 128 or 256 might be more reasonable. For read-only access the buffer amount zero. Depending in the microcontroller and SDK used, a device handle for the FLASH driver is required: for example on a Kinetis this requires an extra of 100 bytes. For others like the LP845 no handle is needed, as the ROM routines are used for flash programming.

On a Kinetis, a read-only configuration adds around 6 KByte of code size (-O0, no optimization), and making it read-write adds an extra of 3 KByte (-O0). Adding shell support adds an extra 1 KByte: so this means that a maximum of 10 KByte of FLASH memory is needed, which gets down to about 6 KByte with optimizations turned on. So far less than a full file system support which easily is 30-50 KByte of FLASH memory.

Summary

With this additional glue port for minINI it is possible to use it without a file system. This makes it suitable for smaller embedded systems. It supports a read-only and read-write data storage. It uses direct flash memory read/writes and only in read-write configuration it needs a configurable RAM buffer for writing or extending the data. With this it is easy to add ‘key-value’ pair storage to any embedded system, with the cost of less than 10 KByte of FLASH memory.

You can find the implementation in the McuLib on GitHub.

Happy flashing 🙂

9 thoughts on “Key-Value pairs in FLASH Memory: file-system-less minINI

  1. Hi Erich,

    thank u for the nice article. U mentioned that u provide default values without any space in the configuration area. However, I am wondering if with there is a way to reset e.g. an entire parameter set to some default values if data is corrupted. Or resetting controller gains sometimes heals odd behaviour.

    BR,
    Gerhard

    Liked by 1 person

  2. Hi again. I think I was not precise enough. Normally I occupy space for each application parameter, twice. One variable for the currently calibrated value and one read-only variable for a default value. When I reset data, then I copy the default data to the current data. Can I save the space for the default values with your McuMinINI approach?

    BR

    Like

    • Hi Gerhard,
      yes, you don’t need the default data way.
      In my application each motor has a calibration offset, and the default is zero (parameter 0 below):
      offset = ini_getl((char*)sectionName, NVMC_MININI_KEY_MOTOR_OFFSET, 0, NVMC_MININI_FILE_NAME);
      With this call I that default value 0, if that key is not present/stored/found in the data. That way I don’t need to store the default in the data file at all. But it will return the stored value (of course) if it is present.
      If you need finer control, latest minINI implementation features the methods ini_haskey() and ini_hassection(), so if you want, you can use that to make decisions based on if something is present in the data or not.

      Like

  3. Pingback: McuOnEclipse Components: 26-Dec-2021 Release | MCU on Eclipse

  4. How important is to use INI text file in these situations? It could be nice if you need to analyze the content on a PC (open a text editor and read the content), but is it the case?

    Most of the time you will never spy in non volatile data directly, maybe only by a debugger during execution. In this case, why don’t you use a bit-by-bit copy of an internal C structure?

    Liked by 1 person

  5. Pingback: LoRaWAN with NXP LPC55S16 and ARM Cortex-M33 | MCU on Eclipse

  6. Pingback: DIY Split-Flap Display | MCU on Eclipse

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.