Multi-Core Symmetric Multi-Processing (SMP) with FreeRTOS

Posted on by

Maybe you are using a multi-core device in your projects, but have not tapped into multi-core usage yet? FreeRTOS V11.0 is out, and the big news is that it has finally Symmetric Multi-Processing (SMP) integrated into the mainline. This greatly simplifies FreeRTOS usage, as I finally can use the same RTOS for my SMP targets and boards, and I can easily switch between single-core and multi-core applications.

Dual-Core Boards running with FreeRTOS

Outline

With SMP, multiple cores of same type are running on the processor. For example dual ARM Cortex-M0+ on the Raspberry Pi RP2040 or a dual ARM Cortex-M33 on the NXP LPC55S69.

As for FreeRTOS SMP support e.g. for RP2040 or for ESP32, one had to use dedicated forks of the RTOS. Now with the version V11.0.0 of FreeRTOS, the SMP support has been integrated back into the mainline. The V11 of FreeRTOS already has been integrated with Processor Expert. V11 with SMP support is now part of the McuLib too.

In this article I’ll show how to enable SMP support and how to use it.

Single-Core vs. Multi-Core FreeRTOS

The following picture shows the difference between FreeRTOS running on a single-core or in a multi-core environment:

The scenario is very common one with shared memory and interrupts between the cores:

  1. Each core is running an ‘idle‘ task which gets executed if no other task on that core is running.
  2. The scheduler is run by the tick interrupt, the context switch interrupt and by the idle task. The pxCurrentTCB points to the currently executing task context. In case of multi-core this is an array of pointers instead a single pointer.
  3. The tasks lists (e.g. the list of ready tasks) is shared between all cores. In the multi-core case a task can be ‘pinned’ to a core.
  4. For the multi-core case, the RTOS uses two locks (e.g. spin-locks, one for tasks and one for interrupts) for its critical section.

FreeRTOS Configuration

As usual, configure FreeRTOS in FreeRTOSConfig.h or through compiler options. The mandatory option for SMP is to specify the number of cores available, e.g.:

/* Set configNUMBER_OF_CORES to the number of available processor cores. Defaults to 1 if left undefined. */
#define configNUMBER_OF_CORES                       (2)

Next, a number of optional configuration items can be set to configure the system. Below are the settings I’m usually use:

#if configNUMBER_OF_CORES>1
  /* When using SMP (i.e. configNUMBER_OF_CORES is greater than one), set
  * configRUN_MULTIPLE_PRIORITIES to 0 to allow multiple tasks to run
  * simultaneously only if they do not have equal priority, thereby maintaining
  * the paradigm of a lower priority task never running if a higher priority task
  * is able to run. If configRUN_MULTIPLE_PRIORITIES is set to 1, multiple tasks
  * with different priorities may run simultaneously - so a higher and lower
  * priority task may run on different cores at the same time. */
  #define configRUN_MULTIPLE_PRIORITIES             0

  /* When using SMP (i.e. configNUMBER_OF_CORES is greater than one), set
  * configUSE_CORE_AFFINITY to 1 to enable core affinity feature. When core
  * affinity feature is enabled, the vTaskCoreAffinitySet and vTaskCoreAffinityGet
  * APIs can be used to set and retrieve which cores a task can run on. If
  * configUSE_CORE_AFFINITY is set to 0 then the FreeRTOS scheduler is free to
  * run any task on any available core. */
  #define configUSE_CORE_AFFINITY                   1

  /* When using SMP (i.e. configNUMBER_OF_CORES is greater than one), if
  * configUSE_TASK_PREEMPTION_DISABLE is set to 1, individual tasks can be set to
  * either pre-emptive or co-operative mode using the vTaskPreemptionDisable and
  * vTaskPreemptionEnable APIs. */
  #define configUSE_TASK_PREEMPTION_DISABLE         0

  /* When using SMP (i.e. configNUMBER_OF_CORES is greater than one), set
  * configUSE_PASSIVE_IDLE_HOOK to 1 to allow the application writer to use
  * the passive idle task hook to add background functionality without the overhead
  * of a separate task. Defaults to 0 if left undefined. */
  #define configUSE_PASSIVE_IDLE_HOOK               0

  /* When using SMP (i.e. configNUMBER_OF_CORES is greater than one),
  * configTIMER_SERVICE_TASK_CORE_AFFINITY allows the application writer to set
  * the core affinity of the RTOS Daemon/Timer Service task. Defaults to
  * tskNO_AFFINITY if left undefined. */
  #define configTIMER_SERVICE_TASK_CORE_AFFINITY    tskNO_AFFINITY
#endif

The shared task list for all the cores has the consequence, that it could be possible that multiple tasks with the same priority are running the same time on different cores. In a normal (single-core) FreeRTOS application, it would not be possible that a task with lower priority would run if there is a task with higher priority running. This can be disabled with setting configRUN_MULTIPLE_PRIORITIES to 0.

See the FreeRTOS web site for more information on the configuration items.

Core Affinity

With configUSE_CORE_AFFINITY I can tell on which core the task shall run. Below an example with two tasks, one to run on core 0 and the other to run on core 1:

BaseType_t res;
TaskHandle_t xHandle0, xHandle1;

  res = xTaskCreate(myCore0Task, "core0_task", 800/sizeof(StackType_t), NULL, tskIDLE_PRIORITY+1, &xHandle0);
  if (res!=pdPASS) {
    for(;;) {} /* error */
  }
  vTaskCoreAffinitySet(xHandle0, (1<<0)); /* run on core 0 */

  res = xTaskCreate(myCore1Task, "core1_task", 800/sizeof(StackType_t), NULL, tskIDLE_PRIORITY+1, &xHandle1);
  if (res!=pdPASS) {
    for(;;) {} /* error */
  }
  vTaskCoreAffinitySet(xHandle1, (1<<1)); /* run on core 1 */

It is possible to directly create a task with an affinity (xTaskCreateAffinitySet, xTaskCreateStaticAffinitySet) or to query the affinity (vTaskCoreAffinityGet). I prefer the vTaskCoreAffinitySet because that way I can create applications with SMP or without it, guarded by configNUMBER_OF_CORES>1.

Starting the Scheduler

The FreeRTOS scheduler gets started as usual with

vTaskStartScheduler();

FreeRTOS uses the concept of a ‘primary’ core (usually core 0, this is as well which is running the vTaskStartScheduler()). The primary core is the one which handles the RTOS tick timer interrupt and which starts all the secondary cores. The primary core runs as well the FreeRTOS IDLE task which is basically the ‘RTOS owned task’ and runs if no other task is running. The secondary cores run an ‘passive’ IDLE task too.

RTOS owned Interrupts

The RTOS only requires a tick interrupt (for tick counting) and interrupt(s) to make context switch. Depending on the port/hardware they can be assigned to a core, or by default it keeps assigned to core 0 (the primary core).

Debugging

It is possible to do single-core-debugging with a multi-core FreeRTOS application. For example, using Visual Studio Code and the cortex-debug extension for the RP2040 use

"device": "RP2040_M0_0"

for core 0 or

"device": "RP2040_M0_1"

for the other core.

To debug both cores, use a ‘chained’ configuration, an example below for J-Link (launch.json):

{
            "name": "Core_0 J-Link",
            "type": "cortex-debug",
            "request": "launch",
            "servertype": "jlink",
            "serverpath": "${env:PICO_SEGGER_PATH}/JLinkGDBServerCL",
            "cwd": "${workspaceRoot}",
            "executable": "${command:cmake.launchTargetPath}",
            "armToolchainPath": "${env:PICO_TOOLCHAIN_PATH}",
            "device": "RP2040_M0_0",
            "interface": "swd",
            "runToEntryPoint": "main", // "_reset_handler" or "main"
            "postLaunchCommands": [
                "monitor semihosting enable",
                "monitor semihosting ioclient 3", // 1: telnet (port 2333); 2: gdb; 3: both telnet and gdbclient output
            ],
            "rtos": "FreeRTOS",
            "svdFile": "${env:PICO_SDK_PATH}/src/rp2040/hardware_regs/rp2040.svd",
            // Multicore
             "chainedConfigurations": {
                "enabled": true,
                "waitOnEvent": "postInit",
                "detached": true,
                "lifecycleManagedByParent": true,
                "launches": [ // Array of launches. Order is respected
                    {
                        "name": "Core_1", // Name of another configuration
                        "folder": "${workspaceRoot}"
                    }
                ]
            },
        },
        {
        "name": "Core_1",
        "type": "cortex-debug",
        "request": "attach",
        "servertype": "jlink",
        "serverpath": "${env:PICO_SEGGER_PATH}/JLinkGDBServerCL",
        "cwd": "${workspaceRoot}",
        "executable": "${command:cmake.launchTargetPath}",
        "armToolchainPath": "${env:PICO_TOOLCHAIN_PATH}",
        "device": "RP2040_M0_1",
        "interface": "swd",
        "rtos": "FreeRTOS",
    },

It works the same way for other debug probes (e.g. CMSIS-DAP NXP LinkServer) or OpenOCD. Below is my configuration for a CMSIS-DAP debug connection:

{
        "name": "CMSIS-DAP Cortex-Debug",
        "type": "cortex-debug",
        "request": "launch",
        "servertype": "openocd",
        "serverpath": "${env:OPENOCD_PATH}/openocd",
        "serverArgs": [
            "-c adapter speed 5000",
            "-c set USE_CORE 0", // TIMER stops, see https://github.com/raspberrypi/picoprobe/issues/45
        ],
        "cwd": "${workspaceRoot}",
        "executable": "${command:cmake.launchTargetPath}",
        "armToolchainPath": "${env:PICO_TOOLCHAIN_PATH}",
        "device": "RP2040",
        "configFiles": [
            "interface/cmsis-dap.cfg",
            "target/rp2040.cfg"
        ],
        "runToEntryPoint": "main", // "_reset_handler" or "main"
        "rtos": "FreeRTOS",
        // Multicore
        "numberOfProcessors": 2,
        "targetProcessor": 0,
        "chainedConfigurations": {
            "enabled": false,
            "waitOnEvent": "postInit",
            "detached": false,
            "lifecycleManagedByParent": true,
            "launches": [ // Array of launches. Order is respected
                {
                    "name": "Core_1 OpenOCD",  // Name of another configuration
                    "folder": "${workspaceRoot}"
                }
            ]
        },
    },
    {
        "name": "Core_1 OpenOCD",
        "type": "cortex-debug",
        "request": "attach",
        "servertype": "openocd",
        "serverpath": "${env:OPENOCD_PATH}/openocd",
        "executable": "${command:cmake.launchTargetPath}",
        "device": "RP2040",
        "rtos": "FreeRTOS",
        "targetProcessor": 1,
    },

In the case of J-Link, there are multiple instances of J-Link GDB server running. For CMSIS-DAP and OpenOCD (I’m using xpack-openocd-0.12.0-1), the sharing happens at the GDB server.

Stopped Cores in VS Code Debugger

Troubleshooting and Limitations

As the official SMT support in FreeRTOS is new, not everything is perfect, with the example of VS Code (cortex-debug v1.12.1, Microsoft Embedded Tools v0.7.0):

  • SMP thread awareness in the call stack is currently not working, because expected RTOS symbols are different. Single-Core FreeRTOS has a variable pxCurrentTCB pointing to the current task. With SMP, it is now an array pxCurrentTCBs[].
ERROR: Mandatory symbol pxCurrentTCB not found.
  • Thread list views (showing all threads in the system) is not working for SMP (“failed to match any supported RTOS”).
  • Ending the RTOS is only implemented in single core mode, not in SMP mode yet.

In single-core mode, views and debugging is working as expected.

Summary

The officially added SMP support in FreeRTOS is great and good news, making multi-core development simpler and easier. FreeRTOS SMP debugging support is limited in the current stage, but I expect that this will improve soon with more developers using SMP in their projects.

Happy multi-coring 🙂

Links

4 thoughts on “Multi-Core Symmetric Multi-Processing (SMP) with FreeRTOS”

What do you think?

This site uses Akismet to reduce spam. Learn how your comment data is processed.