Solar Assistant With EMQX Data Bridge
Overview
My Growatt inverter has been lacking the integration to control the charging of the battery from the grid in Home Assistant. The feature exists in it’s own app and platform but it’s a real pain in the back side having to do it every day.
Solar Assistant
The inverter has a Virtual Power Plant (VPP) mode which allows something else to change the settings like charging or exporting limits. I believe the intention for this mode is to allow governing bodies to limit import and export if the grid is stressed, however don’t quote me.
Using this feature, Solar Assistant taps into the coms port whether that is via USB or some serial port to change the functionality of the inverter. It also provides other features such as reports and graphing however that is not my focus.
I have it installed and running on a Raspberry Pi 4 for my inverter using the beta release, it allows control of charging, discharging and the inverter modes. The basics of what I need.
Out of the box, there is also Home Assistant support using MQTT. This requires you to run an MQTT broker which is available in the Add-on store.
In my case, I use EMQX so the instructions for setting up a MQTT bridge is slightly different and that was the troubles began.
MQTT Bridge
I suspect I was getting message loops but in order to understand the problem and how I solved it, it’s important to understand what a bridge is. I will assume you know what MQTT is.
A bridge is a connection between two or more MQTT brokers. This pattern is best practice to allow segregation of MQTT brokers and somewhat reduce dependencies. In my example, Solar Assistant has it’s own MQTT server using Mosquitto. All data published from Solar Assistant goes into Mosquitto. All of that is self contained and managed within Solar Assistant’s web interface.
EMQX is used by Home Assistant. The main purpose is for IOT devices that support MQTT can publish it’s data and Home Assistant will pick these up and make them available as sensors. Home Assistant may publish messages back to affect changes to the IOT devices.
These two are separate and independent of each other however I need to connect to the two so that Home Assistant can talk to Solar Assistant. This is where a bridge comes in. It joins two (or more) MQTT brokers together. If the bridge connection is severed, the clients that use the data from each MQTT server will be disrupted but most importantly the two MQTT brokers can still function without each other.
EMQX Data Bridge
Setting up data bridge in EMQX is very simple and can be done from the web front end. The only issue I can across was the dashboard for the data bridge was not showing any activity. A restart of EMQX server fixed this.
Turning on egress (outgoing) to make it a bi-directional data bridge caused a messages to loop. This was obvious when messages received count on the data bridge statistics page was going up in the 1000s rather than 100s.
Turning off (and keeping ingress on) stopped this from happening.
You can see in the egress settings, Bridge Mode is enabled so it’s supposed to detect looping messages. Turning this off means Home Assistant can see the current values but cannot change any settings.
Workaround
To prevent the data bridge loop, a filter can be introduced to ensure only the messages which changes Solar Assistant settings are sent from EMQX to Solar Assistant’s Mosquitto. Unfortunately, there’s no easy way (that I can see) to dynamically look for these topics and therefore use them in a filter.
EMQX uses a rule engine and these rules can be setup to filter the messages.
Solar Assistant has two main topics we’re interested in: homeassistant and solar_assistant. Using an MQTT client, you can connect and see the data.
The homeassistant topic is straight forward read-only information. The solar_assistant is where the filter needs to be applied.
Most of the information in solar_assistant are read-only information. However, within certain sub topics there are sub topics that are changed by Home Assistant to allow you to set modes, turn schedules on and off, etc and would need to go back to Solar Assistant for these changes to take effect.
Reading from left to right for the solar_ssistant topic:
-
1. Messages are published inside Solar Assistant’s Mosquitto broker.
2. EMQX data bridge will pull the information from Solar Assistant into EMQX. The EMQX will hold these in a staging topic ready for processing(filtering).
3. EMQX rules are triggered upon a new message in the staging topic.
3a. If the topic and data received matches the rule, the data is re-published inside EMQX to the solar_assistant topic.
3b. If the topic and data received does not meet pass the rule, the message is dropped and goes no further.
4. Home Assistant will update it’s entity information when a message lands in the EMQX solar_assistant topic.
The purple highlights for the sub topic set are the ones which needs to go back to Solar Assistant in order for changes made in Home Assistant to take effect. These message should only go from EMQX to Solar Assistant and filtered out if it comes back from Solar Assistant to EMQX.
Home Assistant Data Bridge
Solar Assistant has split topics: one for Home Assistant, another is Solar Assistant commands and data.
I split the topics to have separate data bridges. This was initial done for debugging purposes but the workaround described later means this method will reduce the processing overhead.
The Home Assistant is a read-only (no bi-directional messaging required) for discovery purposes (if enabled in Solar Assistant and Home Assistant).
The main changes from the Solar Assistant/above settings are:
- Remote MQTT Broker – Topic: homeassistant/#
In summary, this completes the top half of the diagram.
Solar Assistant Ingress Data Bridge
Either use an existing or create a new data bridge. Enable ingress for incoming messages. The changes needed from the previous data bridge are as follows:
- Remote MQTT Broker – Topic: solar_assistant/#
- Local MQTT Broker – Topic: staging/${topic}
The remote now only subscribs to solar_assistant and all sub topics. The messages will be placed in staging top level topic and the rest of the structure will be mirrored. For example, Solar Assistant (remote) topic:
solar_assistant/inverter_1/battery_first_slot_1_enabled/state
would go to:
staging/solar_assistant/inverter_1/battery_first_slot_1_enabled/state
in EMQX.
This completes the data from Solar Assistant to the staging topic.
Solar Assistant Rule
In EMQX 5, go to Integration > Rules and create a new rule.
Give it any name you want. The rule will be triggered when a data bridge receives a message. The right hand side under Events tab has a help by selecting Use bridge on the data bridge created for Solar Assistant’s ingress (above). This will define the text between FROM and WHERE in the screenshot below.
Next is the tedious part: finding the topics that Home Assistant can publish a message to change settings. I suspect this is different depending on your battery/inverter. Pierre gave me this command which is independent of your solar setup:
mosquitto_sub -h 10.0.0.100 -v -t '#' | grep command_topic
If you’re using MQTT Explorer, search for command_topic
In the homeassistant sub topics, all the entities that allow changes from Home Assistant have a command_topic which tells Home Assistant where to publish the requested changes from Home Assistant.
I’d suggest test with one before adding the rest. Taking the command_topic value, add them to the WHERE clause and exclude them. For each one you add after the first, add AND to chain the exclusions together.
Next, add an action to republish. Anything that matches the rule will be have it’s message re-sent from staging/solar_assistant to solar_assistant topic where Home Assistant can pick them up.
Save the rule (maybe a restart of EMQX) and you should see Solar Assistant’s data updating in Home Assistant.
The setup is nearly complete and completes the lower part of the diagram.
EMQX to Solar Assistant
The final part is to get the messages Home Assistant sends from settings change to Solar Assistant. For each command_topic, create a data bridge that is egress (out going) only.
This time, the topics are hard coded for both local (EMQX) and remote (Solar Assistant via Mosquitto).
This completes the lower left side of the diagram.
Repeat from the Workaround for each command topic.
Summary
This was a real pain but once up and running, it works relatively well. I have noticed the changes from Home Assistant not taking hold and reverting back to the current settings. A restart of Mosquitto in Solar Assistant works around this issue as well. All in all it gets the job done.
I hope Solar Assistant will enable MQTT 5.0 protocol and bridges which should make this workaround redundant or if some can tell me what I have done wrong it would be much appreciated.
