UPDATED: 24-09-2021

Having implemented OPC UA PubSub (Publisher/Subscriber) functionality in our SDK for Java gave us a great opportunity to update our Azure demo to utilize these latest features. Our earlier work with this project is described in two previous blog posts: one and two.

In our previous iteration, we used a Microsoft OPC publisher-module to transfer data from an OPC UA server to Azure. Since Azure’s IoT Hubs also allow direct connections using the MQTT protocol, we wanted to replace that component with our own PubSub publisher. As our old Raspberry Pi powered weather station was also starting to show its age, we decided to take it down and replace it with a simulated data source.

NOTE: This implementation requires OPC UA SDK for Java Version 4.6.0

Sending data to Azure with MQTT

Azure IoT Hub provides endpoints allowing IoT devices to communicate with the cloud using the MQTT protocol. While the IoT Hub is not a fully-featured MQTT broker, it still supports receiving OPC UA PubSub messages. Connecting to the IoT Hub’s events allows further processing of the data sent in the device-to-cloud messages.

OPC UA Publisher

OPC UA PubSub is a new communication mechanism defined as an alternative to the basic OPC UA Client/Server communication. PubSub enables a few different alternative transport protocols. MQTT is suited for cloud communications and is widely supported by most cloud products.

OPC UA Publisher is an application that is configured to send fixed datasets to MQTT Brokers, for example, which can then distribute it to several subscribers.

Setting up an OPC UA MQTT publisher that connects to an Azure IoT Hub using our SDK for Java is easy: the SDK includes a sample publisher that can be used to get started quickly. Simply providing the publisher with the right credentials corresponding to a device defined in the IoT Hub makes it ready to send JSON encoded OPC UA data using an MQTT connection secured with TLS. In the SamplePublisherServer, this can be done by starting the server with the following command line parameters:

>samplepublisherserver.bat 
  --address ssl://<IoT Hub hostname>:8883 
  --encoding json 
  --username <IoT Hub hostname>/<device id>/?api-version=2018-06-30 
  --password <device SAS token> 
  --client-id <device id> 
  --queue-name devices/<device id>/messages/events/ 
  --metadata-queue-name devices/<device id>/messages/events/
  --non-reversible-json

Type it all in one line, and in Linux and macOS, use samplepublisherserver.sh instead of .bat. Replace <IoT Hub hostname> with the actual hostname of your IoT Hub instance and <device id> with the ID of the device as it is identified in Azure. <device SAS token> is the IoT Hub Security Token aka ‘Shared Access Signature (SAS)’ of the device.

The Publisher will now send messages in the standard OPC UA PubSub JSON format. By default, the sample publisher will include OPC UA network headers that could be used to filter out some of the data, but with Azure, we need to use the --non-reversible-json argument, which will make the publisher send plain name/value pairs, for example as

[{
  "MyLevel":{
    "Value":73.0,
    "SourceTimestamp":"2021-07-07T17:36:29+0300"
  },
  "MyLevelDisplayName":{
    "Value":"MyLevel"
  }
}]

.NET Core Event handler

Using the Azure Event Hubs client library, we created a .NET core application with a BackgroundService to receive and process device messages through the IoT Hub Event Hub compatible endpoint.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
		string consumerGroup = "azure";

		// Create a blob container client that the event processor will use 
		BlobContainerClient storageClient =
				new BlobContainerClient(_blobStorageConnectionString, <CONTAINER_NAME>);

		// Create an event processor client to process events in the event hub
		EventProcessorClient processor =    
				new EventProcessorClient(storageClient, consumerGroup, _eventHubConnectionString, <EVENT HUB NAME>);

		// Register handlers for processing events and handling errors
		processor.ProcessEventAsync += ProcessEventHandler;
		processor.ProcessErrorAsync += ProcessErrorHandler;
		_logger.LogInformation("Starting event processor");
		// Start the processing
		await processor.StartProcessingAsync(stoppingToken);

}

Received messages are processed into a compatible format and then sent to our Power BI dataset push API-endpoint as a POST request.

async Task ProcessEventHandler(ProcessEventArgs eventArgs)
{
		var deviceMessagesJsonString = Encoding.UTF8.GetString(eventArgs.Data.Body.ToArray());

		List<DeviceMessage> deviceMessages;
		try
		{
				deviceMessages = JsonSerializer.Deserialize<List<DeviceMessage>>(deviceMessagesJsonString);
		}
		catch(Exception e)
		{
				_logger.LogError("failed to deserialize into normal device message");
				return;
		}

		var result = new List<MyLevel>();
		foreach (var message in deviceMessages)
		{
				//Only process recent messages as PowerBi push-datasets have limit of 1 req/second
				if (message.MyLevel != null && eventArgs.Data.EnqueuedTime.AddSeconds(20) > DateTime.UtcNow)
				{
						result.Add(message.MyLevel);
				}
		}

		if (result.Count == 0)
		{
				return;
		}
		var content = new StringContent(
				JsonSerializer.Serialize(result),
				Encoding.UTF8,
				"application/json");

		var client = _clientFactory.CreateClient();
		var res = await client.PostAsync(
				<POWERBI_PUSH_URL>,
				content);
		_logger.LogInformation(res.StatusCode.ToString());
}

We used Azure App Service to host the application. Full instructions on how to use the Event Hubs client library to receive and process events can be found here.

Power BI

Power BI provides various means to turn your data into different kinds of real-time graphs, dashboards, and reports that can be embedded into either your own applications or other Microsoft services such as SharePoint. Since real-time updating dashboards cannot be shared to users who do not own a Power BI Pro license without additional cost to us, we have decided to instead generate a static graph in Power BI that can be embedded to a website with a simple HTML tag. Publish to the web graph data is cached and updated roughly every hour, meaning it won’t necessarily provide the most recent datapoints when viewed on our webpage. The graph generated from our data can be viewed here.

Conclusion

We have shown that our OPC UA SDK for Java is capable of publishing OPC UA data directly to Azure, from where it can be sent to Power BI to be visualized. While our small demonstration remains relatively basic, the implementation can be extended to cover far more complex use cases.

Do you want to know more?

If you are interested in developing your own Azure or other cloud-based systems with OPC UA connectivity, feel free to contact us. We can provide you with software tools and professional services that enable fast development in this rapidly growing new market.

You can send us an email at sales@prosysopc.com or use the contact form.