The story behind
I had a project that needed to be done with ESP-IDF, an ESP32 framework. To encrypt partitions and use unique private client MQTT keys, I needed to flash every board independently, changing settings for each. I discovered a tip that I’ll explain for mass flashing in production. Whether it’s an ESP32 or other boards, if they use CMakeLists, you can do it.
The enlightenment:
First I wanted to make a settings.conf and then add placeholder to values that changes every-time I flash the board the bellow example is composed of encryption key topics connection pin and led pin and all these change every time I flash the code. settings.conf this settings.conf is just an example the real settings.conf is bigger
LED_PIN=35 CNX_PIN=15 STATE_TOPIC="607f1f77bcf86cd799439011/state" CNX_TOPIC="607f1f77bcf86cd799439011/online" MONITOR_TOPIC="607f1f77bcf86cd799439011/monitor" ESP_TOUCH_V2_KEY="YFFnqqnbJtyl0g9G"
idf_component_register(SRCS "main.c"
"provision.c"
"mqtt.c"
INCLUDE_DIRS "."
)
component_compile_options(-Wno-error=format= -Wno-format)
file(READ "settings.conf" CONFIG_FILE_CONTENTS)
# Split the file contents into lines
string(REGEX REPLACE "\n$" "" CONFIG_FILE_CONTENTS "${CONFIG_FILE_CONTENTS}")
string(REPLACE "\n" ";" CONFIG_FILE_LINES "${CONFIG_FILE_CONTENTS}")
# Parse each line to get the variable name and value
foreach(CONFIG_FILE_LINE ${CONFIG_FILE_LINES})
string(REGEX MATCH "([^=]+)=(.*)" CONFIG_FILE_LINE_MATCH ${CONFIG_FILE_LINE})
set(CONFIG_VARIABLE_NAME ${CMAKE_MATCH_1})
set(CONFIG_VARIABLE_VALUE ${CMAKE_MATCH_2})
# Strip any surrounding double quotes from the value
string(REGEX REPLACE "^\"" "" CONFIG_VARIABLE_VALUE "${CONFIG_VARIABLE_VALUE}")
string(REGEX REPLACE "\"$" "" CONFIG_VARIABLE_VALUE "${CONFIG_VARIABLE_VALUE}")
# Add the variable as a compile definition
add_compile_definitions(${CONFIG_VARIABLE_NAME}="${CONFIG_VARIABLE_VALUE}")
message(${CONFIG_VARIABLE_NAME}="${CONFIG_VARIABLE_VALUE}")
endforeach()
Devil’s in the details: (Explanation)
idf_component_register:
This function is used to register a component with the ESP-IDF build system. It specifies the source files (main.c, provision.c, mqtt.c) and the include directories (.).
component_compile_options:
This command sets compile options for the component. In this case, it disables the treating of certain warnings as errors (-Wno-error=format=), which means that the compiler will not stop the build process if these warnings are encountered, and it will simply emit them as warnings.
file(READ "settings.conf" CONFIG_FILE_CONTENTS):
This command reads the contents of the file settings.conf into the variable CONFIG_FILE_CONTENTS.
string(REGEX REPLACE "\n$" "" CONFIG_FILE_CONTENTS "${CONFIG_FILE_CONTENTS}"):
This command removes the newline character at the end of CONFIG_FILE_CONTENTS. This is done to ensure that each line read from the file is properly formatted.
string(REPLACE "\n" ";" CONFIG_FILE_LINES "${CONFIG_FILE_CONTENTS}"):
This command replaces all newline characters in CONFIG_FILE_CONTENTS with semicolons, effectively splitting the contents into a list of lines stored in CONFIG_FILE_LINES.
foreach(CONFIG_FILE_LINE ${CONFIG_FILE_LINES}):
This command iterates over each line in CONFIG_FILE_LINES.
string(REGEX MATCH "([^=]+)=(.*)" CONFIG_FILE_LINE_MATCH ${CONFIG_FILE_LINE}):
This command uses a regular expression to match each line against the pattern variable_name=variable_value. The matched parts are stored in CONFIG_FILE_LINE_MATCH.
set(CONFIG_VARIABLE_NAME ${CMAKE_MATCH_1}):
This command sets the variable CONFIG_VARIABLE_NAME to the first matched group, which is the variable name.
set(CONFIG_VARIABLE_VALUE ${CMAKE_MATCH_2}):
This command sets the variable CONFIG_VARIABLE_VALUE to the second matched group, which is the variable value.
string(REGEX REPLACE "^\"" "" CONFIG_VARIABLE_VALUE "${CONFIG_VARIABLE_VALUE}"):
This command removes any leading double quote from CONFIG_VARIABLE_VALUE.
string(REGEX REPLACE "\"$" "" CONFIG_VARIABLE_VALUE "${CONFIG_VARIABLE_VALUE}"):
This command removes any trailing double quote from CONFIG_VARIABLE_VALUE.
add_compile_definitions(${CONFIG_VARIABLE_NAME}="${CONFIG_VARIABLE_VALUE}"):
This command adds a compile definition for each variable in the format variable_name="variable_value".
message(${CONFIG_VARIABLE_NAME}="${CONFIG_VARIABLE_VALUE}"):
This command prints each variable and its value to the console.
Overall, this script reads a configuration file (settings.conf), parses it line by line, extracts variable names and values, and adds them as compile definitions for the ESP-IDF build system. This allows the project to be configured at build time based on the contents of the configuration file.
Time to get your hands dirty: (Example)
Let’s create a main.c
#include#ifdef VARIABLE_ONE #define VARIABLE_STRING "VARIABLE_ONE" #elif VARIABLE_TWO #define VARIABLE_STRING "VARIABLE_TWO" #else #define VARIABLE_STRING "No variable defined" #endif int main() { printf("Hello, World! This is %s\n", VARIABLE_STRING); return 0; }
Create the CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
project(HelloWorld)
add_executable(HelloWorld main.c)
file(READ "settings.conf" CONFIG_FILE_CONTENTS)
string(REGEX REPLACE "\n$" "" CONFIG_FILE_CONTENTS "${CONFIG_FILE_CONTENTS}")
string(REPLACE "\n" ";" CONFIG_FILE_LINES "${CONFIG_FILE_CONTENTS}")
foreach(CONFIG_FILE_LINE ${CONFIG_FILE_LINES})
string(REGEX MATCH "([^=]+)=(.*)" CONFIG_FILE_LINE_MATCH ${CONFIG_FILE_LINE})
set(CONFIG_VARIABLE_NAME ${CMAKE_MATCH_1})
set(CONFIG_VARIABLE_VALUE ${CMAKE_MATCH_2})
string(REGEX REPLACE "^\"" "" CONFIG_VARIABLE_VALUE "${CONFIG_VARIABLE_VALUE}")
string(REGEX REPLACE "\"$" "" CONFIG_VARIABLE_VALUE "${CONFIG_VARIABLE_VALUE}")
add_compile_definitions(${CONFIG_VARIABLE_NAME}="${CONFIG_VARIABLE_VALUE}")
endforeach()
And last the settings.conf:
VARIABLE_ONE=Hello
mkdir build cd build
And now compile run change the value in the settings recompile rerun and see the magic happen
cmake .. make ./HelloWorld
## What lies Beyond: I used the previous example in a more sophisticated use case as the config was bigger and not all values need to be change so with a settings.conf and a tool that generates the settings.conf every time I was able to flash thousands of boards.
There is another way I discovered along the way and it was using the memory inside the MCU (nvs) to store the data instead of recompiling each time. But it works already,deployed to production and no problems whatsoever so why bother.
The END :Â
After explaining things in my own way and striving for simplicity, I’m wondering if this marks the end of one chapter or the beginning of a new one. I’m feeling a bit tired now, so I might need to take a break.
Next time maybe I’ll discover something else and write about it. For now I am out.