MQTT Datasource

This section will show you how to create and configure a MQTT Datasource. The MQTT Datasource enables you to connect to an MQTT server and subscribe to messages communicated using the MQTT messaging protocol.

Preparing the sample MQTT broker

  1. For this example we will use the Eclipse Mosquitto Open Source MQTT Broker so please download the adequate version from the Download page and install it on your system. But - for this example - do not install as a service - we will start the service manually to have easy access to the broker’s log.

    Installing Mosquitto WITHOUT the service option
    Figure 1. Installing Mosquitto WITHOUT the service option
  2. Open a Windows command prompt (e.g. by pressing WIN+R), entering 'powershell.exe' and pressing Enter).

  3. In the new command prompt window navigate to the Mosquitto installation folder, e.g.

    PS> cd 'C:\Program Files\mosquitto\'

  4. Then start the Mosquito broker in verbose mode.

    PS> ./mosquitto.exe -v

    The broker will start logging to the command prompt window. For the sake of later reference, we will call this window the 'Broker Command Prompt'. Later, when it’s no longer needed, the broker can be stopped by pressing CTRL+C.

    .The Mosquitto Broker is running
    Figure 2. The Mosquitto Broker is running
  5. Open another command prompt window and navigate to the Mosquitto installation folder.

  6. There, publish a test message to the Mosquitto broker by entering the following line. For later reference, this window will be called the 'Pubplisher Command Prompt'.

    PS> ./mosquitto_pub.exe -t "MyTopic" -m "This is my message."

  7. As soon as you hit the Enter key, the Broker Command Prompt Window will display a number of lines, where the broker acknowledges the reception of the message you sent.

    Publishing a message to the Mosquitto Broker
    Figure 3. Publishing a message to the Mosquitto Broker

MQTT Datasource Configuration

  1. In DataStudio, first create a local Connector. Then, select the Connector, right-click Admin  New  External Interface  Datasource from the Context menu add a new datasource. In the Create Object wizard, go to the Server Type page, select "MQTT" from the drop-down menu and enter "MQTT" as the Subscriber Name. Click Create to create the Datasource.

    MQTT Datasource - Server Type
    Figure 4. MQTT Datasource - Server Type
  2. Select the Datasource in the I/O Model and go to the Connection Parameters property compound. Set the Broker Address to the Host Name or IP address of the machine running the MQTT broker, 'localhost' for this example.

    Set the Broker Port to the port number of the MQTT broker. In this example we will use the default port number which is 1883.

    MQTT Datasource - Connection Parameters
    Figure 5. MQTT Datasource - Connection Parameters
  3. The Topic property in the Subscription Data property compound specifies the (sub-)tree of topics which can be subscribed from the MQTT broker. To do so …​

    1. Click on the + icon to add another entry and for this example set its Topic property to '#', and Quality Of Service to '0'.

      For subscriptions, two wildcard characters are supported
      • The '#' character represents a complete sub-tree of the hierarchy and thus must be the last character in a subscription topic string. Example: 'SENSOR/1/\#' - This will match any topic starting with SENSOR/1/, such as SENSOR/1/TEMP and SENSOR/1/HUMIDITY.

      • The '+' character represents a single level of the hierarchy and is used between delimiters. Example: 'SENSOR/\+/TEMP' - This will match SENSOR/1/TEMP and SENSOR/2/TEMP.

      MQTT Datasource - Subscription Data
      Figure 6. MQTT Datasource - Subscription Data

A successful connection is indicated by both state and communication indicators being green. This also means that the Datasource has successfully subscribed to all specified topics and is now waiting for notifications.

Receiving Messages

  1. Go to the Publisher Command Prompt Window and send another message to the broker.

    PS> ./mosquitto_pub.exe -t "Another-Topic" -m "This is the first message which will be received by the system."

    In DataStudio an IO-Item corresponding to the new topic will be created. The message will be displayed in the Faceplate of the Property Panel.

    The new topic underneath the MQTT Datasource and the Message in the faceplate
    Figure 7. The new topic underneath the MQTT Datasource and the Message in the faceplate

Sending Messages to the MQTT Broker

For sending a message to an MQTT Broker from the system, an MQTT Publisher IO-Item object is required.

Creating an MQTT Publisher IO-Item

To create an MQTT Publisher IO-Item …​

  1. Right click on the MQTT Datasource object in the I/O Model tree and select Admin  New  External Interfaces  IO-Item.

  2. In the Create Object wizard, give the new IO-Item an Object Name. For this example use 'MQTT Publisher for Topic-1'

  3. Also set the Item ID property to the same value as the Object Name.

    The Common Property Compound
    Figure 8. The Common Property Compound
  4. Go to the Item Type compound and select 'MQTT'.

  5. In the Topic property, specify the name of the topic to which this IO-Item is supposed to subscribe to. For this example set it to 'Topic-1'.

  6. For Item Type select 'Publisher'.

    The Item Type Property Compound
    Figure 9. The Item Type Property Compound
  7. Click Create to create the new IO-Item in the I/O Model.

    The new IO-Item will appear in the I/O Model in red state. The state will turn green when the first message is successfully sent.

Sending messages from the IO-Item to the MQTT broker

Once the MQTT Publisher IO-Item has been created, it will send a message to the MQTT broker every time its value changes. To test this …​

  1. Select the IO-Item in the I/O Model and double click the value are in the Faceplate.

  2. In the drop down list of the Write Value dialog select 'String'. Then enter a message to the input field. Finally click on Write.

    A new IO-Item will be created for the new Topic in the IO-Item since the Datasource was configured to accept all topics from the MQTT Broker, using the '#' wildcard for the Topic property in the Subscription Data compound.

    Writing a value to the Faceplate to publish an MQTT message
    Figure 10. Writing a value to the Faceplate to publish an MQTT message
  3. Select this new IO-Item in the I/O Model and check the faceplate of the property panel. The value which you entered in the previous step should be displayed there.

    The published message was received
    Figure 11. The published message was received

Callbacks

With Callbacks Lua Scripting can be used to pre-process incoming and outgoing data or to create custom actions which are triggered when certain MQTT related events occur. Action Callbacks (On Connect, On Disconnect and On Delivery') defined on Datasource level apply to the Datasource object itself while Pre-processing Callbacks (Preprocess Outgoing and On Message Receive) on the same level apply to all subordinate IO-Items. Individual Callbacks on IO-Item objects allow specific data pre-processing and actions for each individual IO-Item.

By default the respective Callback properties provide empty functions, illustrating the parameter profiles.

General Callbacks

General Callbacks are located in the Callbacks section of the MQTT Datasource object.

Callback Properties in MQTT Datasource Objects
Figure 12. Callback Properties in MQTT Datasource Objects

General Pre-Processing of Incoming Messages

The script in the On Message Receive property is executed whenever the system receives data from the MQTT Broker. Scripts located in this property can be used to modify the payload before it is forwarded to the IO items.

General Pre-Processing of Outgoing Messages

The script in the Preprocess Outgoing property is executed whenever data is to be sent to the MQTT Broker. It can be used to modify the payload before it is sent to the MQTT Broker.

Custom Actions For Connection Establishments

The script in the On Connect property is executed whenever a connection between the Datasource object and an MQTT Broker can be (re-)established and can be used to inform the system about the state of the connection to the MQTT Broker.

Custom Actions For Connection Ends

The script in the On Disconnect property is executed when the connection between the Datasource object and the MQTT Broker is interrupted and can be used to inform the system about the state of the connection to the MQTT Broker.

General Custom Actions For Confirmed Delivery of Sent Messages

The script in the On Message Delivery property is executed whenever the MQTT Broker confirms that it has received a message from the Datasource.

Individual Pre-Processing & Individual Custom Actions

Besides the general Callbacks which are defined on Datasource level and applied to all subordinate IO-Items, also specific Callbacks can be defined in the properties of each individual IO-Item.

Individual Pre-Processing of Incoming Data

MQTT Subscriber IO-Items can process incoming data individually before it is stored in the system. To automatically apply a Lua script to data received by a specific Subscriber Item, enter the script in the On Message Receive property which is located in the Item Type > Subscriber Properties properties compound of this IO-Item.

Pre-processing incoming data on IO-Item level takes place after the data was passed on from the Datasource object, where it may already have been pre-processed on Datasource level after it was received from the MQTT broker.
Callback Properties on Subscriber IO-Items
Figure 13. Callback Properties on Subscriber IO-Items

Here is an example for how VQT data is extracted from a JSON input:

local json = require"rapidjson"

return function(numid, mid, topic, payload, qos, retain)
	local targetObjectPath = nil -- the subscriber IO-Item itself will be used
  local vqt = json.decode(payload)
  return targetObjectPath, vqt.V, vqt.Q, vqt.T
end

Individual Pre-Processing of Outgoing Data

Data pre-processing for MQTT Publisher IO-Items can be similarly implemented as a Lua function in the Preprocess Outgoing Message property hosted in the Item Type > Publisher Properties property compound of the respective IO-Item.

Pre-processing outgoing data on IO-Item level takes place before the data is passed on the Datasource object, where it can then be further pre-processed before it is sent to the MQTT broker.
Callback Properties on Publisher IO-Items
Figure 14. Callback Properties on Publisher IO-Items

Individual Custom Actions for Confirmed Delivery of Sent Messages

The On Message Delivery property in an MQTT Publisher IO-Item’s Item Type > Publisher Properties property compound allows to provide a Lua function which is executed every time the MQTT Broker confirms that it has received the data from this specific Publisher IO-Item. There are no values which can be returned from this function.