MQTT with lwip and NXP FRDM-K64F Board

In the area of IoT (Internet of Things), one obvious need is to have a way to send and receive data with an internet protocol. MQTT (or Message Queue Telemetry Transport) is exactly like that: a light-weight Machine-to-Machine communication protocol. With the MQTT protocol a microcontroller (or ‘client’) can send data and/or subscribe to data. For example to the Adafruit.IO:

Adafruit MQTT IO Feed

Adafruit MQTT IO Feed

Overview

In this article I show the basic steps to get MQTT running on the NXP FRDM-K64F board using MCUXpresso IDE, lwip and MQTT. lwip ois a small and open source TCP/IP stack which is widely used. To keep things very simple in this first post, I’m using it in bare-metal (no RTOS) mode with no encryption/security. The principle applies to any IDE/toolchain, as long there is a lwip port available for your board and IDE/toolchain. I’m using the MCUXpresso IDE as it nicely integrates with the MCUXpresso SDK which includes a lwip port for the FRDM-K64F.

I’m using the following software and tools:

  1. MCUXpresso IDE v10.0.0 b344
  2. MCUXpresso SDK for FRDM-K64F v2.2.0
  3. lwip v2.0.0

MQTT

MQTT is a very cool connectivity protocol on top of TCP/IP with a subscribe/publish messaging transport. If you don’t know MQTT, then I recommend have a read here:

Broker

To use MQTT, I need a server or ‘broker’. Eclipse hosts the Mosquitto project (https://mosquitto.org/) which is probably one of the most used MQTT brokers. You should be able to find plenty of tutorials about installing Mosquitto on Linux, Raspberry Pi or even Windows.

Having a local broker is great for starting a MQTT project, as you can easily debug it and you get things up and running with a local broker, and then you can switch to a broker/provider in the internet. I have used the following tutorial to install Mosquitto on Windows 10:

https://sivatechworld.wordpress.com/2015/06/11/step-by-step-installing-and-configuring-mosquitto-with-windows-7/

Running Mosquitto on Windows

Running Mosquitto on Windows

Adafruit.io Broker

I love stuff from Adafruit! And they host a very useful MQTT broker, see https://learn.adafruit.com/mqtt-adafruit-io-and-you/overview.

To send and receive data, I have to use a user name with a password (the AIO key below). πŸ’‘ If using the MQTT port 1883, information is sent in clear text which is definitely not something I want. Instead, port 8883 with TLS v1.2 shall be used! In this tutorial I’m assuming a local broker behind a firewall so using port 1883 is ok (but only in this case!).

Adafruit IO Settings

Adafruit IO Settings

MQTT Client Tools

It is very useful to have a client tool installed on the host machine. Have a read at http://www.hivemq.com/blog/seven-best-mqtt-client-tools and choose one. What I am using is the MQTT.fx (http://www.jensd.de/apps/mqttfx/) utility which has a nice GUI to monitor the broker status:

MQTT.fx Broker Status

MQTT.fx Broker Status

To use MQTT.fx with Adafruit.io, see https://cdn-learn.adafruit.com/downloads/pdf/desktop-mqtt-client-for-adafruit-io.pdf.

Creating lwip Project

I started my MQTT project from one of the lwip example projects which come with the MCUXpresso SDK:

Import SDK Examples

Import SDK Examples

And the used the bare metal ‘ping’ example as the base:

lwip ping example

lwip ping example

The example project used in this article is on GitHub (https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/MCUXpresso/FRDM-K64F/FRDM-K64F_lwip_lwip_mqtt_bm).

MQTT Drivers

From the ‘sources’ folder of that project, you need

  • mqtt.h: interface to mqtt.c
  • mqtt.c: MQTT implementation on top of lwip
  • mqtt_opts.h: configuration header file

lwip Initialization

The following code is a standard (bare metal) lwip initialization:

  1. Initialize board pins and clocks
  2. Initialize timer for lwip (time_init())
  3. Configure network interface addresses (own (static) address, netmask and gateway)
  4. lwip initialization (lwip_init())
int main(void) {
 struct netif fsl_netif0;
 ip4_addr_t fsl_netif0_ipaddr, fsl_netif0_netmask, fsl_netif0_gw;

 SYSMPU_Type *base = SYSMPU;
 BOARD_InitPins();
 BOARD_BootClockRUN();
 BOARD_InitDebugConsole();
 /* Disable SYSMPU. */
 base->CESR &= ~SYSMPU_CESR_VLD_MASK;

 time_init();

 IP4_ADDR(&fsl_netif0_ipaddr, configIP_ADDR0, configIP_ADDR1, configIP_ADDR2, configIP_ADDR3);
 IP4_ADDR(&fsl_netif0_netmask, configNET_MASK0, configNET_MASK1, configNET_MASK2, configNET_MASK3);
 IP4_ADDR(&fsl_netif0_gw, configGW_ADDR0, configGW_ADDR1, configGW_ADDR2, configGW_ADDR3);

 lwip_init();

 netif_add(&fsl_netif0, &fsl_netif0_ipaddr, &fsl_netif0_netmask, &fsl_netif0_gw, NULL, ethernetif_init, ethernet_input);
 netif_set_default(&fsl_netif0);
 netif_set_up(&fsl_netif0);

 PRINTF("\r\n************************************************\r\n");
 PRINTF(" MQTT Bare Metal example\r\n");
 PRINTF("************************************************\r\n");
 PRINTF(" IPv4 Address : %u.%u.%u.%u\r\n", ((u8_t *)&fsl_netif0_ipaddr)[0], ((u8_t *)&fsl_netif0_ipaddr)[1],
 ((u8_t *)&fsl_netif0_ipaddr)[2], ((u8_t *)&fsl_netif0_ipaddr)[3]);
 PRINTF(" IPv4 Subnet mask : %u.%u.%u.%u\r\n", ((u8_t *)&fsl_netif0_netmask)[0], ((u8_t *)&fsl_netif0_netmask)[1],
 ((u8_t *)&fsl_netif0_netmask)[2], ((u8_t *)&fsl_netif0_netmask)[3]);
 PRINTF(" IPv4 Gateway : %u.%u.%u.%u\r\n", ((u8_t *)&fsl_netif0_gw)[0], ((u8_t *)&fsl_netif0_gw)[1],
 ((u8_t *)&fsl_netif0_gw)[2], ((u8_t *)&fsl_netif0_gw)[3]);
 PRINTF("************************************************\r\n");

 DoMQTT(&fsl_netif0);
 for(;;) {}
 return 0;
}

The MQTT application logic is implemented in DoMQTT() function.

Connect to MQTT Broker

The following code shows how to connect with lwip to the MQTT broker, adopted from the lwip 2.0.2 code.

The address of the mqtt_client_t structure is passed from the caller:

mqtt_client_t mqtt_client;
mqtt_do_connect(&mqtt_client);

which calls mqtt_do_connect():

static void mqtt_do_connect(mqtt_client_t *client) {
 ip4_addr_t broker_ipaddr;
 struct mqtt_connect_client_info_t ci;
 err_t err;

 IP4_ADDR(&broker_ipaddr, configBroker_ADDR0, configBroker_ADDR1, configBroker_ADDR2, configBroker_ADDR3);

 /* Setup an empty client info structure */
 memset(&ci, 0, sizeof(ci));

 /* Minimal amount of information required is client identifier, so set it here */
 ci.client_id = configMQTT_CLIENT_NAME;
 ci.client_user = configMQTT_CLIENT_USER;
 ci.client_pass = configMQTT_CLIENT_PWD;

 /* Initiate client and connect to server, if this fails immediately an error code is returned
   otherwise mqtt_connection_cb will be called with connection result after attempting
   to establish a connection with the server.
   For now MQTT version 3.1.1 is always used */
 err = mqtt_client_connect(client, &broker_ipaddr, MQTT_PORT, mqtt_connection_cb, 0, &ci);

 /* For now just print the result code if something goes wrong */
 if(err != ERR_OK) {
   printf("mqtt_connect return %d\n", err);
 }
}

The example above uses a static client address. The client information structure is filled with a client name, a password and user name (both can be NULL), and then it connects to the broker with a connection callback (mqtt_connection_cb).

Connection Callback

In the connection callback, if the connection gets accepted, it sets up three callback: one for incoming publish callback (mqtt_incoming_publish_cb()), one for incoming data (mqtt_incoming_data_cb()) and one for subscription requests (mqtt_sub_request()):

static void mqtt_connection_cb(mqtt_client_t *client, void *arg, mqtt_connection_status_t status) {
 err_t err;

 if(status == MQTT_CONNECT_ACCEPTED) {
   printf("mqtt_connection_cb: Successfully connected\n");

   /* Setup callback for incoming publish requests */
   mqtt_set_inpub_callback(client, mqtt_incoming_publish_cb, mqtt_incoming_data_cb, arg);

   /* Subscribe to a topic named "subtopic" with QoS level 1, call mqtt_sub_request_cb with result */
   err = mqtt_subscribe(client, "subtopic", 1, mqtt_sub_request_cb, arg);

   if(err != ERR_OK) {
     printf("mqtt_subscribe return: %d\n", err);
   }
 } else {
   printf("mqtt_connection_cb: Disconnected, reason: %d\n", status);

   /* Its more nice to be connected, so try to reconnect */
   mqtt_do_connect(client);
 }
}

Below are example implementation of the three callbacks:

/* The idea is to demultiplex topic and create some reference to be used in data callbacks
 Example here uses a global variable, better would be to use a member in arg
 If RAM and CPU budget allows it, the easiest implementation might be to just take a copy of
 the topic string and use it in mqtt_incoming_data_cb
*/
static int inpub_id;
static void mqtt_incoming_publish_cb(void *arg, const char *topic, u32_t tot_len) {
 printf("Incoming publish at topic %s with total length %u\n", topic, (unsigned int)tot_len);

 /* Decode topic string into a user defined reference */
 if(strcmp(topic, "print_payload") == 0) {
   inpub_id = 0;
 } else if(topic[0] == 'A') {
   /* All topics starting with 'A' might be handled at the same way */
   inpub_id = 1;
 } else {
   /* For all other topics */
   inpub_id = 2;
 }
}

static void mqtt_incoming_data_cb(void *arg, const u8_t *data, u16_t len, u8_t flags) {
 printf("Incoming publish payload with length %d, flags %u\n", len, (unsigned int)flags);

 if(flags & MQTT_DATA_FLAG_LAST) {
   /* Last fragment of payload received (or whole part if payload fits receive buffer
   See MQTT_VAR_HEADER_BUFFER_LEN) */

   /* Call function or do action depending on reference, in this case inpub_id */
   if(inpub_id == 0) {
     /* Don't trust the publisher, check zero termination */
     if(data[len-1] == 0) {
       printf("mqtt_incoming_data_cb: %s\n", (const char *)data);
     }
   } else if(inpub_id == 1) {
     /* Call an 'A' function... */
   } else {
     printf("mqtt_incoming_data_cb: Ignoring payload...\n");
   }
 } else {
   /* Handle fragmented payload, store in buffer, write to file or whatever */
 }
}

static void mqtt_sub_request_cb(void *arg, err_t result) {
 /* Just print the result code here for simplicity,
 normal behaviour would be to take some action if subscribe fails like
 notifying user, retry subscribe or disconnect from server */
 printf("Subscribe result: %d\n", result);
}

Publishing Data

To publish data, below is an example function. Replace the example topic “erichs/f/test” with your own topic.

static void my_mqtt_publish(mqtt_client_t *client, void *arg) {
 const char *pub_payload= "abcd";
 err_t err;
 u8_t qos = 2; /* 0 1 or 2, see MQTT specification */
 u8_t retain = 0; /* No don't retain such crappy payload... */
 err = mqtt_publish(client, "erichs/f/test", pub_payload, strlen(pub_payload), qos, retain, mqtt_pub_request_cb, arg);
 if(err != ERR_OK) {
   printf("Publish err: %d\n", err);
 }
}

A callback (mqtt_pub_request_cb()) is provided. In my case below I print an error code if it fails:

/* Called when publish is complete either with success or failure */
static void mqtt_pub_request_cb(void *arg, err_t result) {
 if(result != ERR_OK) {
   printf("Publish result: %d\n", result);
 }
}

Summary

With MQTT I can publish/subscribe data on the internet. It is light-weight, open source and makes it ideal for any IoT kind of applications. Using it with the FRDM-K64F and lwip took me a while to get things working (I still face some issues in combination with FreeRTOS, so this is likely subject of a follow-up post). Another thing which I have not covered here is the need for secure connections: using an unencrypted MQTT port 1883 is not something to be used for sensitive data. Instead, port 8883 with SSL/TLS should be used. Yet another reason for a follow-up post I think?

Happy MQTTing:-)

Links

 

Advertisements

31 thoughts on “MQTT with lwip and NXP FRDM-K64F Board

  1. Great article Erich, very interested to see the encrypted version …
    Which stack are u going to use ? Maybe WolfSSL ?

    Regards, Luca

    Like

      • Hi Erich,

        Yes, I’ve seen it now the GitHub projetc : surely the mbed TLS has less concerns about licensing ( open source Apache 2.0 or GNU Public License Version 2.0 ) .

        Is it the first time are you using mbed TLS or have you already used in some other projects ?

        Regards and thanks !

        Like

  2. Hi Erich,
    I wish you to implement it with succes …

    ( Yes, also because it could be very useful for my projects … (: )

    Like

  3. Pingback: Introduction to Security and TLS (Transport Security Layer) | MCU on Eclipse

  4. Pingback: Enable Secure Communication with TLS and the Mosquitto Broker | MCU on Eclipse

  5. Pingback: Easter Weekend Apple Juice Brined Pulled Pork Smoked on Beech Wood | MCU on Eclipse

  6. Pingback: Tutorial: Secure TLS Communication with MQTT using mbedTLS on top of lwip | MCU on Eclipse

  7. Pingback: Tuturial: mbedTLS SLL Certificate Verification with Mosquitto, lwip and MQTT | MCU on Eclipse

  8. Hi Erich,

    Nice post! Very useful.
    I have experienced some issues with this mqtt lib.
    If I publish MQTT messages with a not too high frequency (10Hz) it works properly.
    Log:
    mqtt_publish: Publish with payload length 89 to topic “TEST”
    mqtt_output_send: tcp_sndbuf: 8760 bytes, ringbuf_linear_available: 97, get 27,4
    mqtt_tcp_sent_cb: Calling QoS 0 publish complete callback

    If the frequency is higher, appears some problems at write/output part.
    Log:
    mqtt_publish: Publish with payload length 93 to topic “TEST”
    mqtt_publish: Publish with payload length 93 to topic “TEST”
    mqtt_publish: Publish with payload length 91 to topic “TEST”

    Have you experienced something like this?
    Are this lib limited by publish frequency?

    Thanks for this post again!

    Like

    • Sometimes, I have received some different logs which leads to a board’s block (doesn’t publish anymore and doesn’t seems to works–>this leads me to push reset button).
      Log:
      mqtt_cyclic_timer: Timer should not be running in state 0
      mqtt_tcp_recv_cb: client->conn == pcb

      If someone has any idea, would be fantastic!

      Like

    • Hi Josep,
      thanks :-). I need to look into this. Are you using the latest version from GitHub as the project has been evolved a bit after that article has been published.

      Like

      • Ok. I will stay tuned to any update on your project.
        I have the premonition that this could be due to printf via serial that slow down the running code.

        Like

      • Hi Josep,
        yes, printf is very likely the reason for this. I had not a chance to try it out myself with higher frequency. But you could try to switch off the printf output?

        Like

      • Hi Josep,
        I created a simple test with publishing at 50 Hz and 100 Hz, and have not faced any problems, even with the debug messages turned on, at least with the latest version I run which is on GitHub.
        But you have to make sure that you call ethernetif_input(netifp); /* Poll the driver, get any outstanding frames */
        sys_check_timeouts(); /* Handle all system timeouts for all core protocols */
        frequent enough.

        Like

  9. Hello, Erich
    Thank you for all of your work regarding Kinetis MCU’s – it’s very helpful and interesting data!

    I’ve started with FRDM-K64F devboard and lwip ping example before going further ahead to MQTT protocol. Ping application worked fine at the first glance, however it goes to hard fault after some random time. I’m using an infinite ‘ping local_address’ command in linux console for testing. I found that ENET_ReadFrame function in fsl_enet.c contains memcpy function (line 942) which seems to use a wrong offset during some circumstances thus becomes a reason of memory corruption.
    I still cannot find which situation leads to this fail, maybe you’ve come across similar bug during testing?

    Best regards, Sergey

    Like

    • Hi Sergey,
      I have not seen the failure you describe. But I saw that the TCP driver provided by NXP in the SDK v2.x is not fully thread and RTOS aware. What I ended up is serving TCP from a single RTOS thread (not multiple ones) to solve that problem. In earlier SDK versions it was possible to use a special driver mode to avoid that problem, see the comments section in https://mcuoneclipse.com/2015/10/28/tutorial-lwip-with-the-freertos-and-the-freescale-frdm-k64f-board/
      I hope this helps,
      Erich

      Like

      • Thanks for the link, I’m going to add FreeRTOS later too.

        Though the problem appears even with a baremetal example.

        I’ve just found that everything works great if semihosting is disabled. Board has been running during the whole night and still operational.
        Seems that ethernet buffer overflows during processing time-consuming semihosting printf operations or something like that..

        I also got some odd behavior during debug session. Firstly I’ve used default CMSIS-DAP debug and it wasn’t possible to exit from debug session without repowering the board. I’ve found your article regarding j-link / openSDA then switched to this firmware and now the problem is gone.

        Later on I’ve discovered some CPU freezes after pausing during debug session when using redlib/semi. Only board reset / re-powering was able to fix this issue.
        I’ve tried to switch to newlib and it seems to work ok now. Except for hard fault issues when semihosting is on -)

        Like

      • Did you disable semihosting in the debugger? Semihosting is pretty evil in an embedded environment. Great for a quick demo as you don’t need a terminal or real UART connection, but should not be used in a real application. It might be that the hard fault comes from a stack overflow, so make sure your MSP (main stack pointer size) and space is large enough, as the semihosting interrupt adds to the interrupt load. And yes, it will delay things too.

        Like

      • Stack and heap size has been set to 4Kb during testing.
        Need to check out MSP size, thank you.
        And yes, semihosting is disabled in debugger configuration..

        I’m going to use it more carefully now -)

        Like

  10. Hello, Erich

    I’ve got some strange issues during debugging sessions. I’m using latest mcu expresso (with NXP toolchain) + frdm-k64f + segger opensda 2.0. I switched to windows 10 after some strange debugger behaviour during work on Ubuntu 17 o/s to make sure it’s just an incompatibility issue.

    But I’ve just realized that problem is still here. I’ve tried to use your bare-metal mqtt+lwip example, then modified application level to constantly send a mqtt messages to server. No problems with debugging at all.
    Then I just merged this code with a freescale example for evalution of on-board accelerometer. And I found some issues during debugging session. Most of the time after I press ‘run’ button the board hangs with no indication of problem. Even code stepping (step over actually) leads to that issue sometimes. Only board reset is make sense in that case. No hardware faults. I double checked a stack and heap size, but I don’t use FreeRTOS and it seems that heap/stack are still large enough for lwip sockets.
    When I’ve commented out some parts of the code debugger just stopped after ‘run’ at some part of tcp/ip code though allowing me to make steps further.

    Suddenly I realized that board is works fine after flashing. If debugger only being used to flash the board everything works just great. But if it hangs then only way to make it alive is full repowering via usb connect/disconnect. Also I found there is no multiple GDB stuck in processes.
    I’ve tried to h/w reset board many times after succesfull start-up and found no any sight of troubles. So I suppose it’s not some random code glitches but debugging issue.

    Have you ever come across that kind of issue?

    Best regards, Sergey

    Like

    • Hi Sergey,
      I might have seen this, but not sure. In my case the FreeRTOS Kernel Awareness seemed to have some problems if the project was not a FreeRTOS one. In any case, if you have FreeRTOS kernel awareness enabled, you might give it a try to disable it in the Segger settings.
      The other thing is that I had seen some strange things with older SEGGER versions. You might download a new version from segger.com.

      I hope this helps,
      Erich

      Like

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s