Taking Your M-Bus To The Next Level… MQTT

Taking Your M-Bus To The Next Level… MQTT

Introduction

First of all, why would you do this? Many people own an appartment and want to measure the heat, water, electricity flow to the people living there. Others just want to measure their own consumption a bit more in detail than with their annual bill or writing the numbers down manually. M-Bus is quite a nice bus since it is mostly open source, based on simple serial protocols, supplies your meters (if needed) and there exist quite some cheap interface circuits for it for building it on your own or just buy some USB-mBus-Adapter from aliexpress (this is what I did in the end).

USB-M-Bus-Adapter
M-Bus Heat Meter

BTW: To connect my wmBus meters, I used wmbusmeters from GitHub and also contributed my two Aventies meters (Cold Water Flow and Heat Cost Allocators). You can find the wmbus post here.

Prerequisites

One descision you should make in advance is, where to store all this data and how to store it most efficiently. In my case, I have some Home Assistant running. Before I used iobroker and it was easy to connect the adapter with the mbus adapter, but iobroker somehow got crashing over and over… Nevertheless, influxDB is always a good choice to store measument data. Home Assistent and iobroker use different schemas in influx, but they both keep stuff well arranged. This makes it quite easy to analyze or consume your data with Grafana, node-red or python.

To transfer the data of a non-standard-integration, it is always best to use a rock solid network protocol, even if the deamon collecting the data is running on the same machine. In my case I decided to use MQTT as the transport and collect it with Home Assistant MQTT client. But first steps first…

Another thing to do is, to add your user to the dialup group, to be able to access serial devices. Also check, if /dev/ttyUSB0 is owned by the dialup group and has write access.

M-Bus Reader Software

To read out the mbus, you need some software that understands the M-Bus protocol and can help with addressing. For this purpose, there is some quite old (but rock solid) implementation called libmbus. This is mainly written to be used as a library for your own software, but it includes some basic command line tools, that server quite well for my purpose, at least for initial bring-up of my network.

Here are the commands to get, compile and install it (best run as user):

pi@garagepi:~/GIT/libmbus $ git clone https://github.com/rscada/libmbus.git
pi@garagepi:~/GIT/libmbus $ cd libmbus
pi@garagepi:~/GIT/libmbus $ ./build.sh  # Maybe not necessary
pi@garagepi:~/GIT/libmbus $ ./configure
pi@garagepi:~/GIT/libmbus $ make -j4
pi@garagepi:~/GIT/libmbus $ sudo make install
pi@garagepi:~/GIT/libmbus $ sudo ldconfig

If you did not see errors, that’s all to get started. Now let’s test the net…

Alternative to libmbus

Some people reported problems with libmbus… So you could try an alternative using python: pyMeterBus

It does not support the full feature set of M-Bus (e.g. Extended VIF codes), but it could help debugging problems with libmbus.

Searching for meters

For meters to be found, you need to know the appropriate baud rate. In my case the meters talk with 2400 Baud, so you need to provide this to the mbus-serial-scan or mbus serial-scan-secondary. Nowadays, it totally makes sense, to use secondary addressing. Some meters can be configured to have a primary address, using their secondary address for configuration, but the data rate usually needed is so low, that also secondary addressing is way enough. If you are really interested in how to configure the primary address, let me know and I’ll find out and extend this post.

pi@garagepi:~ $ mbus-serial-scan -b 2400 /dev/ttyUSB0 
Collision at address 0
Found a M-Bus device at address 110
pi@garagepi:~ $ mbus-serial-scan-secondary -b 2400 /dev/ttyUSB0
Found a device on secondary address 0000873987050402 [using address mask 0FFFFFFFFFFFFFFF]
Found a device on secondary address 10223908496A8804 [using address mask 1FFFFFFFFFFFFFFF]
Found a device on secondary address 35001739496A8804 [using address mask 3500173FFFFFFFFF]
Found a device on secondary address 35001740496A8804 [using address mask 3500174FFFFFFFFF]

As you can see, the primary addresses have collisions, since most M-Bus meters a factory set to address 0. Therefore, you should prefer secondary addressing where every meter has it’s unique address.

If you cannot find anything, it totally makes sense to measure the voltage between the two bus lines of the M-Bus. These should show something around 30V and should be stable. If data gets transfered, the voltage will drop a bit and also looks a bit unstable on a standard multimeter until the transfer is finished.

Getting The Data of Some Meters

As an example, I will show, how to get the data from my electricity meter 0000873987050402 (which I use for my electric car’s wall-box since a few weeks).

pi@garagepi:~/GIT/libmbus $ mbus-serial-request-data -b 2400 /dev/ttyUSB0 0000873987050402
<?xml version="1.0" encoding="ISO-8859-1"?>
<MBusData>
    <SlaveInformation>
        <Id>8739</Id>
        <Manufacturer>ALG</Manufacturer>
        <Version>4</Version>
        <ProductName></ProductName>
        <Medium>Electricity</Medium>
        <AccessNumber>2</AccessNumber>
        <Status>00</Status>
        <Signature>0000</Signature>
    </SlaveInformation>
[...]
    <DataRecord id="16">
        <Function>Instantaneous value</Function>
        <StorageNumber>0</StorageNumber>
        <Tariff>0</Tariff>
        <Device>0</Device>
        <Unit>Manufacturer specific</Unit>
        <Value>0</Value>
        <Timestamp>2021-07-06T12:44:20Z</Timestamp>
    </DataRecord>

    <DataRecord id="17">
        <Function>More records follow</Function>
        <Value></Value>
        <Timestamp>2021-07-06T12:44:20Z</Timestamp>
    </DataRecord>
</MBusData>

This already looks fine, but we better could use it, if it is JSON… You could use some XSLT to build your own individual converter, but this would be like taking a nut to crack a sledgehammer. Luckily, there is already a tool for this…: xq (part of yq, using jq) from pyhton pip. We also install mosquitto-clients, since we need it later. To install it, do the following:

pi@garagepi:~/GIT/libmbus $ sudo apt install mosquitto-clients jq python3-pip
pi@garagepi:~/GIT/libmbus $ sudo pip3 install yq
pi@garagepi:~/GIT/libmbus $ mbus-serial-request-data -b 2400 /dev/ttyUSB0 0000873987050402 | xq .
pi@garagepi:~ $ mbus-serial-request-data -b 2400 /dev/ttyUSB0 0000873987050402 | xq .
 {
   "MBusData": {
     "SlaveInformation": {
       "Id": "8739",
       "Manufacturer": "ALG",
       "Version": "4",
       "ProductName": null,
       "Medium": "Electricity",
       "AccessNumber": "5",
       "Status": "00",
       "Signature": "0000"
     },
     "DataRecord": [
       {
         "@id": "0",
         "Function": "Instantaneous value",
         "StorageNumber": "0",
         "Tariff": "1",
         "Device": "2",
         "Unit": "Manufacturer specific",
         "Value": "1800",
         "Timestamp": "2021-07-06T13:09:09Z"
       },
[...]
      {
        "@id": "16",
        "Function": "Instantaneous value",
        "StorageNumber": "0",
        "Tariff": "1",
        "Device": "0",
        "Unit": "Manufacturer specific",
        "Value": "1707949",
        "Timestamp": "2021-07-06T13:09:09Z"
      },
      {
        "@id": "17",
        "Function": "More records follow",
        "Value": null,
        "Timestamp": "2021-07-06T13:09:09Z"
      }
    ]
  }
}

This already looks nice to be transferred to Home Assistant or whatever you want. For the transfer to start in regular intervals, simply create a cronjob and execute a script for all meters to be transferred.

pi@garagepi:~ $ mkdir bin
pi@garagepi:~ $ cd bin
pi@garagepi:~/bin $ touch read_send_meters_mqtt.sh
pi@garagepi:~/bin $ chmod u+x read_send_meters_mqtt.sh
pi@garagepi:~/bin $ vi read_send_meters_mqtt.sh

Now paste the following content into the file, edit it to your needs and you can run it.

#!/bin/bash

ADDRESS_FILE=~/addresses.txt
BAUDRATE=2400
DEVICE=/dev/ttyUSB0
MQTT_HOST=172.22.2.137
MQTT_USER=mqtt
MQTT_PASS=leonardo
MQTT_TOPIC=mbusmeters

if [ ! -f $ADDRESS_FILE ]; then
    mbus-serial-scan-secondary -b $BAUDRATE $DEVICE \
        | sed -e 's/^.*y address \([0-9A-Fa-f]\+\) .*$/\1/' > $ADDRESS_FILE
fi

echo -e "\n $(date)"
echo "Sending data to host $MQTT_HOST as user '$MQTT_USER' using topic '$MQTT_TOPIC/'."

while read ameter
do
    echo -n "Getting data from $ameter..."
    # The sed is for replacing the @ with _ to be able to match on it in HASS templates
    METER_DATA=$(mbus-serial-request-data-multi-reply -b $BAUDRATE $DEVICE $ameter | xq . | sed -e "s/@/_/")
    /usr/bin/mosquitto_pub -h $MQTT_HOST -u $MQTT_USER -P $MQTT_PASS \
        -t $MQTT_TOPIC/$ameter -m "${METER_DATA}"
    BYTCNT=$(echo "$METER_DATA" | wc -c)
    echo "  $BYTCNT bytes sent"
    echo "$METER_DATA" | jq '{ \
        id          : .MBusData.SlaveInformation.Id, \
        manufacturer: .MBusData.SlaveInformation.Manufacturer, \
        medium      : .MBusData.SlaveInformation.Medium, \
        records     : .MBusData.DataRecord | length  }'
done < <(cat $ADDRESS_FILE)

It will create an address.txt file with all meters found. If you added or removed meters, just delete the address.txt and it will be created again.

Here is, what it will output:

The data will be published on MQTT. But be aware, some brokers and debug tools (e.g. MQTT-Explorer) have limited message sizes. With mosquitto_sub -t ... you can always watch the real data. Took me a few minutes to figure out, is was a limitation of MQTT-Explorer.

Making It Work Autonomously

I decided to fetch the data every 5 minutes. Therefore I installed a crontab by executing crontab -e and putting in the following lines:

PATH=$PATH:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
*/5 * * * *   cd /home/pi && bin/read_send_meters_mqtt.sh >>log/mbus_mqtt.log

Adding It To Home Assistant

Making it work in Home Assistant simply requires to add the following lines to sensors.yaml (or below sensors in configuration.yaml when using a monolythic config):

#####################
# MQTT M-Bus meters #
#####################
- platform: mqtt
  name: "Heat Outlet Hot Water"
  unique_id: mbusmeters.10223908496A8804
  state_topic: "mbusmeters/10223908496A8804"
  unit_of_measurement: "kWh"
  value_template: >
    {%- for rec in value_json.MBusData.DataRecord -%}
    {%- if ( 
            ( rec.Function == "Instantaneous value" )
       and  ( rec.StorageNumber == "0" )
       and  ( "kWh" in rec.Unit )
          ) -%}
    {{ rec.Value }}
    {% endif -%}
    {% endfor -%} 

If you want all data from your Electric Meter put into the attributes of the Sensor, this is how it works for algodue meters (with Unit, Tariff and Device elements in some objects).

- platform: mqtt
  name: "EM Garage ENERGY_COUNTER"
  unique_id: mbusmeters.0000873987050402
  state_topic: "mbusmeters/0000873987050402"
  unit_of_measurement: "kWh"
  value_template: >
    {%- for rec in value_json.MBusData.DataRecord -%}
    {%- if ( 
             ( rec.Function == "Instantaneous value" )
        and  ( rec.StorageNumber == "0" )
        and  ( rec._id == "34" )
        and  ( "Wh" in rec.Unit )
           ) -%}
    {{ rec.Value | float / 10000.0 | round(2) }}
    {% endif -%}
    {% endfor -%}
  json_attributes_topic: "mbusmeters/0000873987050402"
  json_attributes_template: |-
    {
      {% for mbd in value_json.MBusData.SlaveInformation -%}
        "{{ mbd | lower }}" : "{{ value_json.MBusData.SlaveInformation[mbd] }}",
      {% endfor -%}
      {%- for drec in value_json.MBusData.DataRecord %}
        {%- if not drec.Function == "More records follow" %}
      "{{ drec.Function | replace(' ','_') }}_
          {%- if drec.Unit is defined -%}
    {{ drec.Unit | replace(' ','_') }}_
          {%- endif -%}
          {%- if drec.StorageNumber is defined -%}
    {{ drec.StorageNumber }}_
          {%- endif -%}
          {%- if drec.Tariff is defined -%}
    {{ drec.Tariff }}_
          {%- endif -%}
          {%- if drec.Device is defined -%}
    {{ drec.Device }}_
          {%- endif -%}
    {{ drec._id }}" : "{{ drec.Value }}"
          {%- if not loop.last -%}
    ,
          {%- endif -%}
        {%- endif -%}
      {% endfor %}
    }

And this is, how it looks like in Home Assistant algodue electricity meter:

aventies water meter:

zenner heat outlet:

This config only captures the instant counter value. I will soon add the config to add the history values as attributes and also add the other values like power, temperature/voltage, flow/current,… Stay tuned.

For the Zenner Zelsius C5 heat outlet meters, I even have a few more detailed documents, kindly provided by the manufacturer:

Also algodue provided some information about their energy counter:

Troubleshooting

If you experience problem with the script, try the following:

  1. Try running the mbus-serial-request-data-multi-reply command on the command line
  2. Delete the addresses.txt (it’s only a cache to speed up the readings)
  3. Check if the script has execution permissions
  4. Change the shebang (first line of script) to #!/bin/bash -x to get addidtional output, what is executed and if it is the same as the working command from (1), if it is not executed at all, it could not find any meters on the bus or your addresses.txt is empty

If this still does not help, write me an Email.

Have fun!

Tell Me What You Think About This Article

Latest Reviews

Tolle Einführung, die mir sehr geholfen hat

Rated 5 out of 5
26. Januar 2022

Ich habe mir erlaubt, dein Script etwas zu erweitern:

#!/bin/bash

# Code Basis stammt von https://the78mole.de/taking-your-m-bus-online-with-mqtt/

# Ich habe es nur ergänzt um alle Datenfelden flexibel zum MQTT Server zu übertragen

# Getestet nur mit einem Wärmezähler. Ich hoffe es felxibel genug auch für andere Geräte

ADDRESS_FILE=“./addresses.txt“

BAUDRATE=2400

DEVICE=/dev/ttyAMA0

MQTT_HOST=HOSTNAME

MQTT_USER=MQTT_USER

MQTT_PASS=MQTT_PASSWORT

MQTT_TOPIC=MQTT_TOPIC

#Basis Informationen

modbus_array[0]=.MBusData.SlaveInformation.Id

modbus_array[1]=.MBusData.SlaveInformation.Manufacturer

modbus_array[2]=.MBusData.SlaveInformation.Version

modbus_array[3]=.MBusData.SlaveInformation.Status

#———————————————————-

while read ameter

do

echo -n „Getting data from $ameter…“

# The sed is for replacing the @ with _ to be able to match on it in HASS templates

METER_DATA=$(mbus-serial-request-data-multi-reply -b $BAUDRATE $DEVICE $ameter | xq . | sed -e ’s/@/_/‘)

/usr/bin/mosquitto_pub -h $MQTT_HOST -u $MQTT_USER -P $MQTT_PASS

-t $MQTT_TOPIC/$ameter -m „${METER_DATA}“

BYTCNT=$(echo „$METER_DATA“ | wc -c)

echo “ $BYTCNT bytes sent“

# Basis Parameter auslesen

for Parameter in „${modbus_array[@]}“

do

ausgabe=$(echo $METER_DATA | jq „$Parameter“ )

#Punkte in / konvertieren für mqtt

Parameter=$(echo $Parameter | sed -e ’s!.!/!g‘)

/usr/bin/mosquitto_pub -h $MQTT_HOST -u $MQTT_USER -P $MQTT_PASS -t $MQTT_TOPIC/$ameter$Parameter -m „$ausgabe“

#Ausgabe des MQTT Pfades und Wertes

echo „$MQTT_TOPIC/$ameter$Parameter -> $ausgabe“

done

#Schleife über alle Data Records

for row in $(echo $METER_DATA | jq ‚.MBusData.DataRecord | keys‘); do

#Leere Datenfelder am Anfang und Ende abfangen

if [[ ${row} != „[“ ]] && [[ ${row} != „]“ ]]

then

#Das Komma am Ende entfernen

id=$(echo „${row}“ | sed -e ’s/,//‘)

#Einen Int Wert daraus machen

nummer=$((„$id“+0))

#Alle Keys (Datenpunkte) auslesen

for name in $(echo $METER_DATA | jq „.MBusData.DataRecord[$nummer] | keys“); do

#Leere Datenfelder am Anfang und Ende abfangen

if [[ ${name} != „[“ ]] && [[ ${name} != „]“ ]]

then

# Ein paar Sonderzeichen aus dem Pfad entfernen

pfad=“.MBusData.DataRecord[$nummer].${name}“

pfad=$(echo $pfad | sed ’s/“//g‘)

pfad=$(echo $pfad | sed ’s/,//g‘)

#Den richtigen Wert aus dem json holen

wert=$(echo $METER_DATA | jq $pfad)

#Umformatieren für eine MQTT Pfad

pfad=$(echo $pfad | sed ’s/.///g‘)

pfad=$(echo $pfad | sed ’s/[/_/g‘)

pfad=$(echo $pfad | sed ’s/]//g‘)

#Zum MQTT Server senden

/usr/bin/mosquitto_pub -h $MQTT_HOST -u $MQTT_USER -P $MQTT_PASS -t $MQTT_TOPIC/$ameter$pfad -m „$wert“

echo „$MQTT_TOPIC/$ameter$pfad –> $wert“

fi

done

fi

done

done < <(cat $ADDRESS_FILE)

Martin

Antwort von MolesBlog

Hallo Martin,

schön, dass es Dir geholfen und gefallen hat.

Danke auch für die Verbesserungsvorschläge. Die werde ich mir mal genauer anschauen und einpflegen.

Grüße,

Daniel (the78mole)

Great tutorial!

Rated 5 out of 5
7. Januar 2022

I never thought I could read 8 M-bus meters in Homeassistant.

Thank you very much for this great job.

Fernando

Taking Your M-Bus To The Next Level… MQTT article

Rated 5 out of 5
27. Juli 2021

Hi,

I have successfully integrated your approach (Taking Your M-Bus To The Next Level… MQTT article) to read data from my water meter, only when launching the read_send_meters_mqtt.sh script does the reading of the information take place not. I think my credentials are wrong. What value should I enter under MQTT_Host? This is the IP address of the broker with which I will receive the data (here Jeedom in a virtual machine) or is it the IP address of the Raspberry Pi with which I read the Mbus values? Same, which MQTT_User and MQTT_Pass values should I enter?

Thanks,

Joël

Joël Stauber
themole

2 Kommentare

Lutz Veröffentlicht am14:38 - 11. Oktober 2022

Hallo, danke für diese Inspiration. Ich würde das gerne an meinen Gas-Sensor anpassen, aber leider hängt es ein wenig, bei dem angepassten Script aus dem Kommentar.
So weit bin ich gekommen:
mbus-serial-request-data-multi-reply -b 2400 /dev/ttyAMA0 10 | xq . | sed -e „s/@/_/“
{
„MBusData“: {
„SlaveInformation“: {
„Id“: „40867341“,
„Manufacturer“: „ELS“,
„Version“: „37“,
„ProductName“: null,
„Medium“: „Gas“,
„AccessNumber“: „9“,
„Status“: „00“,
„Signature“: „0000“
},
„DataRecord“: {
„_id“: „0“,
„Function“: „Instantaneous value“,
„StorageNumber“: „0“,
„Unit“: „Volume (m m^3)“,
„Value“: „1545810“,
„Timestamp“: „2022-10-11T10:56:38Z“
}
}
}
Möglicherweise sind die Leerzeichen in den Werten ein Problem?
Hier die Debug-Ausgabe:
mbusmeter/10///////////////////////////////// -> „00“
++ echo ‚{‚ ‚“MBusData“:‘ ‚{‚ ‚“SlaveInformation“:‘ ‚{‚ ‚“Id“:‘ ‚“40867341″,‘ ‚“Manufacturer“:‘ ‚“ELS“,‘ ‚“Version“:‘ ‚“37″,‘ ‚“ProductName“:‘ null, ‚“Medium“:‘ ‚“Gas“,‘ ‚“AccessNumber“:‘ ‚“26″,‘ ‚“Status“:‘ ‚“00″,‘ ‚“Signature“:‘ ‚“0000″‚ ‚},‘ ‚“DataRecord“:‘ ‚{‚ ‚“_id“:‘ ‚“0″,‘ ‚“Function“:‘ ‚“Instantaneous‘ ‚value“,‘ ‚“StorageNumber“:‘ ‚“0″,‘ ‚“Unit“:‘ ‚“Volume‘ ‚(m‘ ‚m^3)“,‘ ‚“Value“:‘ ‚“1545810″,‘ ‚“Timestamp“:‘ ‚“2022-10-11T12:13:13Z“‚ ‚}‘ ‚}‘ ‚}‘
++ jq ‚.MBusData.DataRecord | keys‘
+ for row in $(echo $METER_DATA | jq ‚.MBusData.DataRecord | keys‘)
+ [[ [ != \[ ]]
+ for row in $(echo $METER_DATA | jq ‚.MBusData.DataRecord | keys‘)
+ [[ „Function“, != \[ ]]
+ [[ „Function“, != \] ]]
++ echo ‚“Function“,‘
++ sed -e s/,//
+ id='“Function“‚
+ echo ‚“Function“‚
„Function“
./rread_send_mbus_2_mqtt.sh: line 72: „Function“+0: syntax error: operand expected (error token is „“Function“+0“)

Für etwas Hilfe wäre ich sehr dankbar.
Gruss Lutz

    themole Veröffentlicht am19:42 - 11. Oktober 2022

    Hallo Lutz,

    mir fehlt da zugegebenermaßen ein wenig Information, z.B. Dein Script rread_send_mbus_2_mqtt.sh, das in Zeile 72 offenbar ein Problem enthält. Das Kommando „keys“ sagt mir nichts und ich kann auch nicht erkennen, was das hier tun soll… Stopfe doch Dein komplettes Skript in https://pastebin.com/ und schicke hier den Link. Dann schaue ich es mir gerne mal an.

    Grüße,
    Daniel

Schreibe einen Kommentar

WordPress Cookie Plugin von Real Cookie Banner