I’m in the final stage of finishing a electrical vehicle (EV) charger controller, which optimizes battery loading using the available PV system: use as much as possible the solar energy and not the grid.

While the controller talks with an Modbus (RS-485) interface to the vehicle charger itself (see Controlling an EV Charger with Modbus RTU), it uses MQTT over WiFi to get information about the available solar energy from HomeAssistant and the Powerwall.

Outline

The hardware is based on a Raspberry Pi Pico W board, 4 WS2812B addressable WRGB LEDs for status, a 4-way+center navigation switch with an OLED as display for status and as user interface. The application runs FreeRTOS, and the GUI is implemented with lvgl.

The Raspberry Pi Pico uses the SDK 1.4.

💡 At writing of this article, the SDK 1.5 just has been made available. I have not tried that new SDK yet.

The PCB has been designed in KiCAD:

To protect the electronics, I have created an enclosure using 3 mm PMMA with the laser cutter:

The unit controls a Heidelberg Energy Control EV charger over Modbus (see Controlling an EV Charger with Modbus RTU).

MQTT is used to talk to the home automation system implemented with HomeAssistant. With the MQTT protocol, the controller receives information about the solar energy produced and how much is used by the building. That way the system can adjust the charging levels to maximize usage of solar energy for the charging process: instead feeding energy to the grid, it is used first to charge the vehicle battery.

All files can be found on Github (see links at the end of this article).

MQTT Broker

First, I need a server to provide me the data (topics). I’m using here HomeAssistant running on a Rasperry Pi 4, but you can use any kind of MQTT broker. In my case I’m using the ‘Mosquitto broker’ in HomeAssistant to publish values from the Tesla Powerwall gateway:

For the MQTT connection, we need the login information for the MQTT broker. This is found behind the ‘Configure’:

Then use ‘Re-Configure MQTT’:

Then you can set or get the username and password:

The Tesla Powerwall status and data is available within the HomeAssistant integration:

To publish the topics with MQTT, you have to edit the /config/configuration.yaml. Below is my configuration which publishes my topics:

# https://www.home-assistant.io/integrations/mqtt_statestream
mqtt_statestream:
  base_topic: homeassistant
  publish_attributes: true
  publish_timestamps: true
  include:
    entity_globs:
      - sensor.powerwall_solar_now
      - sensor.powerwall_load_now
      - sensor.powerwall_site_now
      - sensor.powerwall_battery_now
      - sensor.powerwall_charge
      - binary_sensor.powerwall_charging
      - binary_sensor.powerwall_connected_to_tesla

I recommend verifying the topics, for example with MQTT Explorer:

MQTT Client Files

The MQTT client is using lwIP and the Raspberry Pi Pico SDK. Below are the most important files (see link to GitHub at the end of the article):

• mqtt.c: lwip MQTT functions, based on lwIP example
• dns_resolver.c: custom domain name server resolver with lwIP. Translates host names into IP addresses.
• mqtt_client.c: MQTT client implementation

User names and passwords are not stored in the source files (only dummy values): the real values are stored using MinINI on the device itself.

lwipopts.h

The lwipopts.h configures the lwIP network stack. If using it with MQTT, there are two critical settings to add, otherwise it won’t work:

/* You need to increase MEMP_NUM_SYS_TIMEOUT by one if you use MQTT!
 * see https://forums.raspberrypi.com/viewtopic.php?t=341914
 */
#define MEMP_NUM_SYS_TIMEOUT   (LWIP_NUM_SYS_TIMEOUT_INTERNAL + 1)
#define MQTT_REQ_MAX_IN_FLIGHT  (5) /* maximum of subscribe requests */

The last item is about the number of subscribe requests, and I’m using 5 in my application.

Connect to Broker

To connect to the broker, I need a handle:

static mqtt_t mqtt; /* information used for MQTT connection */

The handle gets initialized as below:

mqtt_client = mqtt_client_new(); /* create client handle */

Connection information is stored inside a struct:

typedef struct mqtt_t {
  mqtt_client_t *mqtt_client;       /* lwIP MQTT client handle */
  DnsResolver_info_t addr;          /* broker lwip address, resolved by DNS if hostname is used */
  unsigned char broker[32];         /* broker IP or hostname string. For hostname, DNS will be used */
  unsigned char client_id[32];      /* client ID used for connection */
  unsigned char client_user[32];    /* client user name used for connection */
  unsigned char client_pass[96];    /* client user password */
  topic_ID_e in_pub_ID;             /* incoming published ID, set in the incoming_publish_cb and used in the incoming_data_cb */
} mqtt_t;

static mqtt_t mqtt; /* information used for MQTT connection */

For the lwIP MQTT connection, I have to resolve first the broker hostname (e.g. “myServer”) to an IP address (e.g. 192.168.0.123). For this, a little DNS resolver is used:

  /* resolve hostname to IP address: */
  if (DnsResolver_ResolveName(mqtt.broker, &mqtt.addr, 1000)!=0) { /* use DNS to resolve name to IP address */
    McuLog_error("failed to resolve broker name %s", mqtt.broker);
    return;
  }

Then I register the connection callbacks:

  /* setup callbacks for incoming data: */
  mqtt_set_inpub_callback(
      mqtt_client, /* client handle */
      mqtt_incoming_publish_cb, /* callback for incoming publish messages */
      mqtt_incoming_data_cb, /* callback for incoming data */
      LWIP_CONST_CAST(void*, &mqtt_client_info) /* argument for callbacks */
      );

For the connection I need a struct with the client information:

static const struct mqtt_connect_client_info_t mqtt_client_info = {
  mqtt.client_id, /* client ID */
  mqtt.client_user, /* client user name */
  mqtt.client_pass, /* client password */
  100,  /* keep alive timeout in seconds */
  NULL, /* will_topic */
  NULL, /* will_msg */
  0,    /* will_qos */
  0     /* will_retain */
#if LWIP_ALTCP && LWIP_ALTCP_TLS
  , NULL
#endif
};

Using this, I can make the connection to the broker:

  /* connect to broker */
  cyw43_arch_lwip_begin(); /* start section for to lwIP access */
  mqtt_client_connect(
      mqtt_client, /* client handle */
      &mqtt.addr.resolved_addr, /* broker IP address */
      MQTT_PORT, /* port to be used */
      mqtt_connection_cb, LWIP_CONST_CAST(void*, &mqtt_client_info), /* connection callback with argument */
      &mqtt_client_info /* client information */
      );
  cyw43_arch_lwip_end(); /* end section accessing lwIP */

Connection Callback

If connection is successful, the connection callback I have provided during mqtt_client_connect() gets called. In that callback I can register to the topics:

static void mqtt_connection_cb(mqtt_client_t *client, void *arg, mqtt_connection_status_t status) {
  const struct mqtt_connect_client_info_t *client_info = (const struct mqtt_connect_client_info_t*)arg;
  LWIP_UNUSED_ARG(client);
  err_t err;

#if MQTT_EXTRA_LOGS
  McuLog_trace("MQTT client \"%s\" connection cb: status %d", client_info->client_id, (int)status);
#endif
  if (status!=MQTT_CONNECT_ACCEPTED) {
    McuLog_error("MQTT client \"%s\" connection cb: FAILED status %d", client_info->client_id, (int)status);
  }
  /* subscribe to topics */
  if (status == MQTT_CONNECT_ACCEPTED) {
    McuLog_trace("MQTT connect accepted");
    err = mqtt_sub_unsub(client,
            TOPIC_NAME_SOLAR_POWER, /* solar P in kW */
            1, /* quos: 0: fire&forget, 1: at least once */
            mqtt_request_cb, /* callback */
            LWIP_CONST_CAST(void*, client_info),
            1 /* subscribe */
            );
    if (err!=ERR_OK) {
      McuLog_error("failed subscribing, err %d", err);
    }
    ...
  } else if (status==MQTT_CONNECT_DISCONNECTED) {
    McuLog_trace("MQTT connect disconnect");
  }
}

The subscribe topics look like this:

/* HomeAssistant Tesla Powerwall topics */
#define TOPIC_NAME_SOLAR_POWER          "homeassistant/sensor/powerwall_solar_now/state"
#define TOPIC_NAME_SITE_POWER           "homeassistant/sensor/powerwall_load_now/state"
#define TOPIC_NAME_GRID_POWER           "homeassistant/sensor/powerwall_site_now/state"
#define TOPIC_NAME_BATTERY_POWER        "homeassistant/sensor/powerwall_battery_now/state"
#define TOPIC_NAME_BATTERY_PERCENTAGE   "homeassistant/sensor/powerwall_charge/state"

Publishing and Data Callbacks

The data for the subscribed topics comes with two callbacks: first with the mqtt_incoming_publish_cb() and then the data with mqtt_incoming_data_cb().

To distinguish between the different topics, I’m using a list of IDs :

typedef enum topic_ID_e {
  Topic_ID_None,
  Topic_ID_Solar_Power,       /* power from PV panels */
  Topic_ID_Site_Power,        /* power to the house/site */
  Topic_ID_Grid_Power,        /* power from/to grid */
  Topic_ID_Battery_Power,     /* power from/to battery */
  Topic_ID_Battery_Percentage,/* battery level percentage */
} topic_ID_e;

The IDs get assigned based on the topic string:

static void mqtt_incoming_publish_cb(void *arg, const char *topic, u32_t tot_len) {
  const struct mqtt_connect_client_info_t *client_info = (const struct mqtt_connect_client_info_t*)arg;

  if (McuUtility_strcmp(topic, TOPIC_NAME_SOLAR_POWER)==0) {
    mqtt.in_pub_ID = Topic_ID_Solar_Power;
  } else if (McuUtility_strcmp(topic, TOPIC_NAME_SITE_POWER)==0) {
    mqtt.in_pub_ID = Topic_ID_Site_Power;
  } else if (McuUtility_strcmp(topic, TOPIC_NAME_GRID_POWER)==0) {
    mqtt.in_pub_ID = Topic_ID_Grid_Power;
  } else if (McuUtility_strcmp(topic, TOPIC_NAME_BATTERY_POWER)==0) {
    mqtt.in_pub_ID = Topic_ID_Battery_Power;
  } else if (McuUtility_strcmp(topic, TOPIC_NAME_BATTERY_PERCENTAGE)==0) {
    mqtt.in_pub_ID = Topic_ID_Battery_Percentage;
  } else { /* unknown */
    McuLog_trace("MQTT client \"%s\" publish cb: topic %s, len %d", client_info->client_id, topic, (int)tot_len);
    mqtt.in_pub_ID = Topic_ID_None;
  }
}

The ID is then used in the data callback:

static void mqtt_incoming_data_cb(void *arg, const u8_t *data, u16_t len, u8_t flags) {
  const struct mqtt_connect_client_info_t *client_info = (const struct mqtt_connect_client_info_t*)arg;
  LWIP_UNUSED_ARG(data);
  unsigned char buf[32];
  int32_t watt;

#if MQTT_EXTRA_LOGS
  McuLog_trace("MQTT client \"%s\" data cb: len %d, flags %d", client_info->client_id, (int)len, (int)flags);
#endif
  if(flags & MQTT_DATA_FLAG_LAST) {
    /* Last fragment of payload received (or whole part if payload fits receive buffer. See MQTT_VAR_HEADER_BUFFER_LEN)  */
    if(mqtt.in_pub_ID == Topic_ID_Solar_Power) {
      GetDataString(buf, sizeof(buf), data, len);
      McuLog_trace("solarP: %s", buf);
      watt = scanWattValue(buf);
      if (watt>=0) { /* can only be positive */
        McuHeidelberg_SetSolarPowerWatt(watt);
      }
    } else if(mqtt.in_pub_ID == Topic_ID_Site_Power) {
      GetDataString(buf, sizeof(buf), data, len);
      McuLog_trace("siteP: %s", buf);
      watt = scanWattValue(buf);
      if (watt>=0) { /* can only be positive */
        McuHeidelberg_SetSitePowerWatt(watt);
      }
    } else if(mqtt.in_pub_ID == Topic_ID_Grid_Power) {
      GetDataString(buf, sizeof(buf), data, len);
      McuLog_trace("gridP: %s", buf);
    } else if(mqtt.in_pub_ID == Topic_ID_Battery_Power) {
      GetDataString(buf, sizeof(buf), data, len);
      McuLog_trace("battP: %s", buf);
    } else if(mqtt.in_pub_ID == Topic_ID_Battery_Percentage) {
      GetDataString(buf, sizeof(buf), data, len);
      McuLog_trace("bat : %s%%", buf);
    } else {
      McuLog_trace("mqtt_incoming_data_cb: Ignoring payload...");
    }
  } else {
    McuLog_trace("mqtt_incoming_data_cb: fragmented payload ...");
    /* Handle fragmented payload, store in buffer, write to file or whatever */
  }
}

With this, an example session then looks like this:

01.01.2020 00:00:00,00 INFO  PicoWiFi.c:99: started WiFi task
01.01.2020 00:00:00,00 INFO  PicoWiFi.c:106: enabling STA mode
01.01.2020 00:00:00,00 TRACE Shell.c:194: started shell task
01.01.2020 00:00:00,22 INFO  McuHeidelberg.c:631: connected with charger
01.01.2020 00:00:00,88 INFO  PicoWiFi.c:117: setting hostname: pico
01.01.2020 00:00:01,88 INFO  PicoWiFi.c:122: connecting to SSID '2.4GHz'...
01.01.2020 00:00:05,60 INFO  PicoWiFi.c:130: success!
01.01.2020 00:00:05,61 INFO  dns_resolver.c:20: 'homeassistant' resolved to 192.168.1.10
01.01.2020 00:00:05,62 INFO  dns_resolver.c:20: 'pool.ntp.org' resolved to 89.111.15.218
01.01.2020 00:00:05,67 INFO  ntp_client.c:42: got ntp response: 11/02/2023 18:17:59
11.02.2023 18:17:59,52 TRACE mqtt_client.c:183: MQTT connect accepted
11.02.2023 18:17:59,57 TRACE mqtt_client.c:114: solarP: 0.0
11.02.2023 18:17:59,58 TRACE mqtt_client.c:121: siteP: 0.2
11.02.2023 18:17:59,58 TRACE mqtt_client.c:128: gridP: -0.006
11.02.2023 18:17:59,58 TRACE mqtt_client.c:131: battP: 0.2
11.02.2023 18:17:59,58 TRACE mqtt_client.c:134: bat : 96%
11.02.2023 18:18:17,02 TRACE mqtt_client.c:128: gridP: -0.036
11.02.2023 18:18:17,02 TRACE mqtt_client.c:131: battP: 0.21
11.02.2023 18:18:17,02 TRACE mqtt_client.c:121: siteP: 0.19

Summary

MQTT is not part of the Pico W SDK 1.4, but with a few tweaks I was able to get it working. Now I have a charger controller talking with MQTT with the HomeAssistant, and I can exchange any kind of data I like.

I hope this helps you to bring MQTT to your Pico W projects too. In any case: check the latest 1.5 SDK as it says:

Added pico_lwip_mqtt library to expose the MQTT app functionality in lwIP.

If you already jumped on 1.5 SDK and using it for MQTT, let me know.

Sources, project files including the enclosure files are available on Github.

Happy MQTTing 🙂

Links

• Project on GitHub: https://github.com/ErichStyger/mcuoneclipse/tree/master/Examples/RaspberryPiPico/pico_Heidelberg
• Getting Started: Raspberry Pi Pico RP2040 with Eclipse and J-Link
• Raspberry Pi Pico SDK: https://github.com/raspberrypi/pico-sdk
• MQTT: https://en.wikipedia.org/wiki/MQTT
• HomeAssistant: https://www.home-assistant.io/
• Heidelberg vehicle charger: Controlling an EV Charger with Modbus RTU
• MQTT Explorer: http://mqtt-explorer.com/
• Tuturial: mbedTLS SSL Certificate Verification with Mosquitto, lwip and MQTT
• Tutorial: Secure TLS Communication with MQTT using mbedTLS on top of lwip
• Energy Crisis in Europe: Optimizing a Building from 4.5 to 2.4 MWh