Cloud Sink with Azure IoT Hub

In this Jumpstart we will create a new Azure IoT Hub, which supports MQTT communications, and configure it. Next we are going to use system:inmation Cloud Sink as a publisher to send a message to Azure IoT Hub using the MQTT protocol.

Prerequisites

To complete the examples in this Jumpstart the following things are required:

Azure account

Azure account which is going to be used should be initially configured, meaning that there already should be a valid Azure subscription and all the necessary permissions to add and manage resources. There also should be at least one Resource Group. More information about Resource Groups is available on the official Microsoft Documentation webpage

Visual Studio CODE (for configuring a device with SAS Token Authentication)
OpenSSL (for configuring a device with X.509 CA Authentication)
system:inmation with Data Studio

Preparing Azure Infrastructure

Login to Azure Portal and click on the button
Find and add a new IoT Hub

Figure 1. Adding a new IoT Hub
Make sure you have the following parameters set using the suggested values or use the below screenshot as a guide:
On the Basics tab enter the following parameters.
- Name - This will be used during the Cloud Sink configuration (SAS Token Authentication) so make a note of it. In this example the following name was used: inmationMqttTest
On the Size and Scale tab enter the following parameters.
- Pricing and scale tier - Any desired option. For this example we will use the Free tier option.
Free tier allows to send up to 8000 messages, which is sufficient for testing purposes

Figure 2. Basic configurational tab

Figure 3. Scale and Size configurational tab
When finished, select the tab and then click and wait until the resource is deployed

In case of encountering any troubles, this official Microsoft Quickstart can be used as an alternative example of how to create a new IoT Hub

Creating a device with SAS Token Authentication

Go to the Azure Portal and select the IoT Hub resource, which was created in Preparing Azure Infrastructure. You can find it in
Find the connection string to configure under the iothubowner Policy ( IoT Hub Shared access policies iothubowner Connection string-primary key ). A connection string should be similar to this:
HostName=inmationMqttTest.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=cHJpbWFyeSBvciBzZWNvbmRhcnkga2V5Ow==
Make a note of it for later, we will need this connection string to generate the SAS Token.

Figure 4. Obtaining Connection String for IoT Hub
Open Visual Studio CODE and go to Extensions tab to install the Azure IoT Tools extension

Figure 5. Azure IoT Tools: a Visual Studio CODE extension
To connect Visual Studio Code to the IoT Hub, open the Explorer tab in Visual Studio CODE and open AZURE IOT HUB. You should see something like this:

Figure 6. Setting Connection String to access IoT Hub through Visual Studio CODE extension

Click on → Set IoT Hub Connection String and insert the Connection String obtained on Azure portal in the previous step into the popup input box.
Now go back to the Azure Portal and select the IoT Hub resource (The one from Preparing Azure Infrastructure).
Click on IoT Devices to create a new device ( IoT Hub IoT devices + New ).

Figure 7. Creating a new IoT Device
Enter a Device ID name and keep a note of it, we will need it later. In this example indevice is used a a name. Set Authentication type to Symmetric key and leave everything else as it is. Click button.

Figure 8. Configuring IoT Device

Return to Visual Studio CODE. Refresh the Device section and your newly created device should appear in the list.

Right-click on it and select Generate SAS Token for Device option.

Figure 9. Generating SAS Token with Visual Studio CODE extension

Insert an expiration time in hours in the input box, for example 24 hours. Generated SAS Token will be automatically placed in the clipboard as well as written to the output console. Make a note of it since it will be needed later for Cloud Sink configuration (SAS Token Authentication). An example of SAS Token is presented below:

SharedAccessSignaturesr=inmationMqttTest.azure-devices.net%2Fdevices%2Findevice&sig=X%2B3r3iqIOu%2B9jPkmcy7pJ496RiPL8scvplo9A%2F18g2k%3D&se=1572964996

When SAS Token exipers, it will no longer be accepted by Azure MQTT Broker and a new one should be generated. It is possible to specify a longer period of time when generating a token, but it is higly recomended to use X.509 Authentification instead.

Now we need to obtain a DigiCert Baltimore Root certificate.

First of all, download certs.c from the Azure IoT SDK for C.

Simple copy-pasting may not work as some extra line-breaker characters might appear and corrupt the file. It is recommended to use browser Save as… command or use any download tool which can accept URL as a file source.

This file contains hardcoded certificate data used for application development. To create a valid certificate, we need to extract certificate data and create a certificate file in a correct way.

Since the instruction given by Microsoft on how to obtain a DigiCert Baltimore Root certificate is very convoluted, there is a help script available as a part of esi-cloud-interface-initializer.lua library, which is available as a part of the system. In order to get the necessary certificate:

Open a new console in DataStudio ( Ctrl+Shift+U )

Run the following lua command:

local path_to_c_file = "C:\\Jumpstart\\certs.c" -- path to certs.c file
return require("esi-cloud-interface-initializer"):CERT(path_to_c_file)

An output like File [C:\Jumpstart\certs.cer] created indicates successful certificacte generation.

Alternitevely this file can be created manually. Using the .c file downloaded earlier, you need to remove all C-related code form it. Also make sure you have -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- lines included, removed the " marks at the beginning and end of every line, and removed the \r\n characters at the end of every line. When the content is modified properly, file extension should be changed from .c to .cer.

Cloud Sink configuration (SAS Token Authentication)

In DataStudio create a new Cloud Sink object by selecting a Connector and right-clicking Admin New History Cloud Sink from the context menu. Give the object a name a click Create to create it in the I/O Model.
Select the Cloud Sink object and in the Object Properties panel, open the Configuration and select MQTT Publisher Interface.
Under MQTT Publisher Parameters fill the Topic property:

Table 1. MQTT Publisher Parameters

Parameter Value Reference Example

Topic

devices/ + [ Device ID ] + /messages/events/

S3:S7

devices/indevice/messages/events/

Table 1. MQTT Publisher Parameters
Parameter	Value	Reference	Example
Topic	`devices/` + [ Device ID ] + `/messages/events/`	S3:S7	devices/indevice/messages/events/

Under Connection Parameters, enter the following values for the parameters:

Table 2. Connection Parameters
Parameter	Value	Reference	Example
Broker Address	[ Hub name ] + `.azure-devices.net`	S2:S3	inmationMqttTest.azure-devices.net
Broker Port	`8883`		8883
Client ID	[ Device ID ]	S3:S7	indevice

Figure 10. Cloud Sink MQTT configuration

Under the Credentials property groups, configure the parameters as instructed below:

Table 3. Connection Parameters
Parameter	Value	Reference	Example
Username	[ Hub name ] + `.azure-devices.net` + [ Device ID ] + `/?api-version=2018-06-30`	S2:S3 S3:S7	inmationMqttTest.azure-devices.net /indevice/?api-version=2018-06-30
Password	[ Generated SAS Token ]	S3:S8	SharedAccessSignature sr=inmationMqttTest.azure-devices.net%2Fdevices%2Findevice& sig=X%2B3r3iqIOu%2B9jPkmcy7pJ496RiPL8scvplo9A%2F18g2k%3D& se=1572964996

Cloud Sink Credentials and SSL parameters

Figure 11. Cloud Sink Credentials and SSL configuration

Under SSL/TLS support import Certificate Authority file

Click Import File on property Certificate Authority and select certs.cer file obtained in this step

Figure 12. Certificate Authority property

Figure 13. Selecting Certificate Authority file

If everything is done correctly, it should be possible to view certificate details by clicking View

Figure 14. Certificate details
Click Apply in the Object Properties to save configuration.

Creating a device with X.509 CA Certificate Authentication

In this tutorial we are going to use self-generated Certificate Authority (CA) certificate. For production environments it is recommended to purchase X.509 certificates from a root certificate authority (CA).

Go to the Azure Portal and select the IoT Hub resource, which was created in Preparing Azure Infrastructure. You can find it in
Click on IoT Devices to create a new device ( IoT Hub IoT devices + New ).

Figure 15. Creating a new IoT Device
Enter a Device ID name and keep a note of it, we will need it later. In this example indevice is used as a name. Set Authentication type to X.509 CA Signed and leave everything else as it is. Click button.

Figure 16. Configuring IoT Device
Download and extract from the archive OpenSSL biniaries. For this example OpenSSL Binaries Win-64 1.1.1d were used.
First we need to generate a new CA (Certificate Authority) root certificate. To do that follow the instructions:
1. Open Command Prompt terminal (Windows Start Windows System Command Prompt ) and change directory to the folder where openssl.exe file is located
  cd "C:\\openssl-1.1.1d-win64"
2. Use the following command to generate a new CA root certificate:
  openssl req -x509 -newkey rsa:2048 -keyout rootKey.pem -nodes -out rootCA.pem
  
  You will be promted to provide some certificate data (such as Country, City, Organisation Name, etc.) Here is the example of how those fields are filled
  
  Country Name (2 letter code) [AU]:DE State or Province Name (full name) [Some-State]:NRW Locality Name (eg, city) []:Cologne Organization Name (eg, company) [Internet Widgits Pty Ltd]:inmation Organizational Unit Name (eg, section) []: Common Name (e.g. server FQDN or YOUR name) []:TEST ROOT CA Email Address []:
  
  As an otput you should obtain two files rootCA.pem (CA root certificate) and rootKey.pem (CA root private key).
  
  Do not close Command Prompt terminal as we will need it later
The created CA certificate needs to be uploded to Azure and then verified. This verification procedure is knowns as Proof of Possession.
1. Login to Azure Portal. Select your IoT Hub resource
2. Upload generated rootCA.pem certificate to Azure ( Iot Hub Certificates + Add ).
  
  Figure 17. Uploading Root Certificate to Azure
3. The certificate should now appear in the certificate list. Click on the newly added certificate to open its properties, next click Generate Verification Code (see Figure 15: Button 1) and copy generated data Verification Code (see Figure 15: Button 2)
  
  Figure 18. Generating verification code and uploading verification file
4. Go back to the Command Prompt terminal used in step 5
5. Create a new verification key by using the following command:
  openssl genrsa -out verification.key 2048
6. Next, we need to create a Certificate Signing Request by executing the following command
  openssl req -new -key verification.key -out verification.csr
  
  When creating the Certificate Signing Request, we again are going to be requested to provide some certificate data. But in this case we need to specify the verification code obtained in step 6.c as the "Common Name" certificate field. Here is the example of how this should look like:
  
  Country Name (2 letter code) [AU]:DE State or Province Name (full name) [Some-State]:NRW Locality Name (eg, city) []:Cologne Organization Name (eg, company) [Internet Widgits Pty Ltd]:inmation Organizational Unit Name (eg, section) []: Common Name (e.g. server FQDN or YOUR name) []:7A69A4702DA903A41C3A5BC5575A8E3F49BEC5E5BA2D4CE1 Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:
7. Now creating the Proof of Possession certificate with the following command:
  openssl x509 -req -in verification.csr -CA rootCA.pem -CAkey rootKey.pem -CAcreateserial -out verificationCert.pem -days 1024 -sha256
  
  As an output you should obtain a new file verificationCert.pem
  
  Do not close Command Propmt terminal as we will need it later
8. Next import the generated verificationCert.pem file (see Figure 15: Button 3) and click button
  
  Figure 19. Root Certificate has been successfully verified
Next task is generating the Device certificate.
1. Use the same Command Prompt terminal we’ve been using through this tutorial (the last time it was used on step 6.g) to create a new Key and Certificate Signing Request
  openssl req -nodes -newkey rsa:2048 -days 365 -keyout device.key -out device.csr
  
  Again, we need to follow the procedure of providing certificate data. In this case we need to specify the Device ID set in step 3 as the "Common Name" certificate field. Here is the example of how this should look like (used name is indevice):
  
  Country Name (2 letter code) [AU]:DE State or Province Name (full name) [Some-State]:NRW Locality Name (eg, city) []:Cologne Organization Name (eg, company) [Internet Widgits Pty Ltd]:inmation Organizational Unit Name (eg, section) []: Common Name (e.g. server FQDN or YOUR name) []:indevice Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:
  
  Also pay attention to the device.key, the Device Key file, which will also be used for Cloud Sink configuration (X.509 CA Authentication) later.
2. And finally we can sign Device certificate with CA root certificate we created in step 5.b by running this command:
  openssl x509 -req -days 365 -in device.csr -CA rootCA.pem -CAkey rootKey.pem -CAcreateserial -out device.crt
  
  As an output we have device.crt certificate, which is going to be used for Cloud Sink configuration (X.509 CA Authentication).
For Cloud Sink configuration we will also need DigiCert Baltimore Root certificate (certs.cer), the same that was used for SAS Token Authenticated devices. This step will explain how to get it.

Cloud Sink configuration (X.509 CA Authentication)

In DataStudio create a new Cloud Sink object by selecting a Connector and right-clicking Admin New History Cloud Sink from the context menu. Give the object a name a click Create to create it in the I/O Model.
Select the Cloud Sink object and in the Object Properties panel, open the Configuration and select MQTT Publisher Interface.

Configure MQTT Publisher Parameters and Connection Parameters:

Table 4. MQTT Publisher Parameters and Connection Parameters
Parameter	Value	Reference	Example
Topic	`devices/` + [ Device ID ] + `/messages/events/`	S5:S3	devices/indevice/messages/events/
Broker Address	[ Hub name ] + `.azure-devices.net`	S2:S3	inmationMqttTest.azure-devices.net
Broker Port	`8883`		8883
Client ID	[ Device ID ]	S5:S3	indevice

Figure 20. Cloud Sink MQTT configuration

Configure Credentials and SSL parameters:

Parameter Value Reference Example

Parameter	Value	Reference	Example
Username	[ Hub name ] + `.azure-devices.net` + [ Device ID ] + `/?api-version=2018-06-30`	S2:S3 S5:S3	inmationMqttTest.azure-devices.net /indevice/?api-version=2018-06-30

Username

[ Hub name ] + .azure-devices.net +

[ Device ID ] + /?api-version=2018-06-30

S2:S3 S5:S3

inmationMqttTest.azure-devices.net

/indevice/?api-version=2018-06-30

For x.509 Authentication algorithm password field stays clear

Under SSL/TLS support import necessary certificates
Following the same procedure of importing a certificate as shown in this step, three certificates have to be imported to the corresponding properties:
- Certificate Authority - DigiCert Baltimore Root certificate (certs.cer) (from step 8)
- Client Certificate - Device certificate (device.crt) (from step 7.b)
- Client Key - Device Key file (device.key) (from step 7.a)
Figure 21. Cloud Sink Credentials and SSL configuration
Click Apply in the Object Properties to save configuration.

Transferring data with Cloud Sink

To check that the interface is working, double-click on Cloud Sink object’s Faceplate and enter a string in the Write Value dialog.

Figure 22. Writing some value to Cloud Sink Item Value
Now check the Azure IoT Hub metric to see that the message has been sent

Figure 23. Azure IoT Hub metric with the number of incoming messages
Alternatively, Azure IoT Hub extension for Visual Studio Code can be used to monitor device messages (the one, which was used for SAS Token generation). To start monitoring a device open AZURE IOT HUB in the Explorer tab, right-click on the necessary device and then select Start Monitoring Build-in Event Endpoint

Figure 24. Enabling message monitoring

Figure 25. An incoming message reported by Azure IoT Hub extension

A single device cannot be connected multiple times to Azure IoT Hub. If a new device with the same Device ID is connected from somewere else, all previous connections will be abandoned.

After deleting a Cloud Sink object, some certificates might be left in the corresponding certificate stores. In order to remove old certificates run the following command in a console:

return require("esi-cloud-interface-initializer"):CLEAN()

Executing this command will not affect any certificates, which are currently being used by other Cloud Sink objects