Custom Messaging Solution for IoT Devices

Why We Decided to Develop Our Own Messaging Solution

In one of our recent IoT projects, we built an API on the server’s side of a device for its interaction with the peripherals. During the project development, we had to face some common challenges. Initially, we tried to meet them using standard networking technologies.

The team tried using HTTP for communication with the IoT system’s web dashboard. For interaction between the server and the client devices (particularly, charging stations), we decided to use MQTT. This lightweight protocol is a common choice for client-server communication in low-bandwidth systems. 

But some charging stations had to use 2G networks with irregular communication and poor throughput. So, the MQTT protocol could not provide a stable connection and efficient data transfer, and the team had to find a better solution for the project. 

The system required something more reliable. To optimize the communication between the components and make the connection with the charging stations more stable, the team decided to create a custom messaging solution.

ISML has much in common with similar IoT messaging protocols. It is also a lightweight solution intended for smart networks that require fast and efficient data exchange. It uses a communication model similar to the messaging mechanism of MQTT and has easy implementation like the POrtable COmponents C++ Libraries (POCO).

However, we can point out several advantages of our custom IoT messaging product that can optimize the interaction of components within your system.

• Documentation. We created a clear and detailed guide to help clients design, deploy, and support their own ISML-based projects. 
• Flexibility. ISML is a cross-platform solution that can be highly customized for various environments. 
• Minimal overhead. The overhead of our solution is lower than that of HTTP and MQTT. The library features a very light architecture and runs over TCP. 
• Scalability. The number of nodes that can be used in your custom systems is only restricted by server capacity. So, it’s a highly scalable solution. 
• Security. ISML can filter incoming messages with a malware protection system. 
• The simplicity of implementation. The library is very developer-friendly. Third parties won’t have any trouble customizing and integrating it into their current C++ code. 
• The simplicity of deployment. Integra Sources will provide you with all the required components, tools, and instructions. So, all you need to deploy ISML is a local or cloud server.

Our messaging library has wide use and can fit a number of projects connected with data exchange and serialization, session management, and inter-networking. ISML is one of the solutions that the Integra engineering team creates within our system and application software development services. We build network applications and protocols that provide smooth communication between the components of IoT and embedded systems.

What is ISML?

ISML is a set of components designed to integrate message-oriented middleware (MOM). MOM is the software that supports communication within a distributed environment. The message-oriented infrastructure of the ISML library facilitates the implementation of a dispatcher, router, and other components related to data delivery.

You can use the library to develop applications with client-server and peer-to-peer (P2P) architectures. ISML provides interfaces to create, send, receive, and store communication data. It is designed for the efficient transmission of binary files. There are several classes used to generate different types of messages and deliver them.

ISML works with sessions so it doesn’t support connectionless protocols, such as UDP and IP. At the present moment, it is compatible with only one network protocol, which is TCP.

However, we singled out the transport layer as a separate class. It helps the client device interact with the peripherals, for example, via RS-485. So, in the future, you can build a data exchange system using any bus and transport protocol - either connectionless or connection-oriented.

Just like similar messaging technologies, ISML uses a publish/subscribe pattern but a slightly different one. For instance, the MQTT protocol uses a broker working as a mediator between nodes, while ISML allows clients and the server to interact directly. 

Additionally, our solution does not require topics because the server has a list of clients subscribed to certain sessions. You can also simplify data transfer by creating a message channel for each session. 

Lastly, you can achieve any security level with ISML as its architecture allows you to add any authentication systems, encryption, and cryptographic protocols.

ISML is a highly flexible and customizable solution that enables users to implement their own logic, leveraging the components of the library.

ISML Components

ISML comprises interrelated components or classes used to implement the functionality of a data exchange system. The basic classes include Session, Transport, SessionManager, and Message.

Session

A communication session introduced as the Session class is one of the key components of the library. It helps identify the messaging client in the system. This class represents a high-level interface for sending and receiving messages.

You can also use Session to store various data necessary for the management of the latest session. The data includes session lifetime, latest activity time, the number of transmitted messages, etc.

The user can expand and customize the list of the stored parameters with the help of a built-in mechanism for the Properties class - a heterogeneous container for data storage.

Session also provides a high-level interface for sending requests and replies. It differs from the standard sending and receiving of messages in its synchronized nature. It allows the requestor to send a message and switch to a waiting mode without changing the execution context.

This mechanism can be helpful for sending confirmations or notifications. It assists in implementing the Remote Procedure Call (RPC) and requests with the use of the messaging system (for example, a request to the network node granting access to the database).

Transport

In an ordinary use case, sessions appear in the system during the setup of the connection with a remote client (on the server’s side) and upon server connectivity (on the client’s side). In both cases, the socket connection gets established regardless of the applied protocol - whether it is TCP or WebSocket.

ISML offers the Transport component that encapsulates all the necessary resources to manage the connection and a part of the logic responsible for sending and receiving messages.

Transport equips the library user with a high-level interface that allows for messaging without reference to all low-level details specific to the used protocol. For example, we use the Asio library that provides an input/output context and a set of objects to work with TCP. The Transport class implements the messaging mechanism on the basis of this protocol hiding the TCP/IP stack routine from the user.

Along with that, Transport is responsible for storing the messages sent and received. It releases users from building their own mechanisms that implement this functionality. The messages are stored in a queue that can operate either in a single-thread system or under concurrency.

Thus Transport makes it possible to work with a queue in a multithreaded environment feeling secure about data or API races. The queue interface eliminates the risk of the races using the optional types. It also excludes all possible methods that could lead to such situations, for example, calculations of the queue size.

Creating a session, you should pass a class instance that implements message Transport. The session uses it to send and receive messages.

SessionManager

ISML contains the SessionManager component that manages sessions and provides the interface to create them. For the component’s usability, you can register a callback in the network sub-system on one of the possible events - connection accepted or connected.

An Acceptor or ConnectionListener is a low-level component of the library. It is responsible for receiving the connection events and notifying SessionManager of the necessity to create a session.

Thus, in case of a successful connection, an instance of the message Transport will be passed to SessionManager. The latter will create a new session and connect it with the specified Transport.

Message and Related Classes

The Message class introduced by ISML is a data unit of a messaging system. Message is a set of fields used to store valuable data that builds up the message body. It also includes helper fields used to identify a message, determine its type and owner.

The MessageFactory class is a special component that creates messages out of the raw data. Building messages outside MessageFactory is unacceptable and will lead to system errors. To create a message, you should know its type and session that acts as a context.

The type of message defines its structure. There can be any number of message types registered in the system. What is important is that they should be unique. A message type itself is an ordinary numerical identifier. Its underlying type is determined by MessageType.

As soon as a message is created, it is then put into the message channel or queue. Dispatcher is the library component that registers handlers for different message types. Thus, handler 1 is invoked to process  MessageType 1, etc.

To create a reply, we should provide MessageFactory with a specific MessageType and a Session.

One numerical identifier is obviously not enough to describe the message structure. ISML comprises the Field class, which is a basic type of message field, and FieldDescriptor. The latter provides an interface to receive additional information about the field, for example, its name, and assists in building the desired field type.

Both components belong to abstract classes, and they can’t serve the purposes mentioned above directly. To store data, you can use the descendant classes - DataField and DataFieldDescriptor respectively. This method is called type erasure. It is used to store any data, including built-in types of the C++ language and user-defined types (UDTs).

FiledDescriptors get united into a message descriptor that is passed to MessageFactory as a prototype. This prototype will be the basis for an instant message of a specific type.

Schematically, we can show it as follows:

So, each message has a descriptor that contains information about the message type (also called a numeric identifier) and its fields. These fields are not stored in the message descriptor itself, but field descriptors are used - they create an actual field of the desired type with the desired name.

Thus, the message descriptor is the prototype of the message, when using information about the message type and fields, messages can be created from its descriptor an unlimited number of times.

In other words, MessageDescriptor contains descriptors of all the fields that the message of this type will include. Thus, Message is a heterogeneous container and it can embrace the fields that store values of any type of data.

Conclusion

The widespread use of the Internet of Things is closely connected with the growth of energy-efficient communication technologies. They make it possible to create networks of smart devices that effectively interact with each other.

Such standards as MQTT, HTTP, and POCO are broadly used in IoT development. However, they are not multipurpose and you can’t implement them in any scenario. Some projects require much more flexible and easy-to-use solutions.

Integra Sources Messaging Library is a custom technology that provides strong and stable communication within a network of connected devices. It can make your project development easy, timely, and cost-friendly. If you want to learn more about our IoT messaging solution or customize it according to your needs, feel free to get in touch with our engineering team.

Copyright © 2021 Integra Sources. All rights reserved.

This library is an internal development of Integra Sources Ltd.

This library is published with usage restrictions.

If you are interested in using this library in your project, please contact us and we will permit you to use it on your personal terms.

License.

ISML on GiTHub