Embedded QT HMI development: why, where, and how. Challenges and case studies.

A human-machine interface (HMI) is a general term for engineering solutions that enable interaction between people and machines, computer systems, or applications.

Why Do We Need an HMI?

Starting and stopping processes, as well as monitoring, can be done remotely, thus reducing the number of errors and the time required for making decisions.

In all their variety of forms, the primary goal of all HMIs is to enable efficient communication between humans and machines in a clear and accessible way. The cornerstone of HMI design is portability and ease of use.

The process of creating an HMI includes the development of hardware and software, as well as the creation of the necessary infrastructure.

The more complex the tasks an HMI faces, the more complex its structure becomes. Sophisticated user interfaces are becoming an integral part of production equipment.

Cross-platform frameworks and GUI toolkits allow developers to create intuitive user interfaces for software of any complexity.

Many cross-platform frameworks, such as .Net, GTK, Xamarin, JUCE, and Flutter, have tools that can be successfully used to develop HMIs of various levels of complexity and for various purposes.

Some frameworks are designed to create applications in a certain area; for example, there are specialized toolkits for automotive production. 

Thus, Kanzi, the software framework for the automotive industry, offers you HMI design and development tools to create advanced multi-OS, multi-display user interfaces. Kanzi experts implement end-to-end solutions for the design, development, and production of automotive human-machine interfaces.

Siemens offers a wide range of SIMATIC HMI products, from configuration software to SCADA systems and software for process visualization. SIMATIC WinCC offers different software options depending on the complexity of the product: Basic, Comfort, and Advanced. WinCC-Advanced is available as a runtime software package with the possibility of additional options.

Qt for HMI Software Development

The Qt framework stands out from other solutions with its performance, versatility, and ease of use. It provides developers with solutions to help meet the growing demands of customers. In our previous article, we highlighted the benefits of the Qt framework and C++ for creating cross-platform applications for embedded systems. HMIs are closely related to embedded systems, and Qt is actively used for HMI software development.

Qt benefits

The Qt framework has undeniable advantages for embedded HMI development:

• Both open licenses and commercial variants are available. To get all the legal and functional benefits of the framework for creating devices, you need a commercial license from The Qt Company. But if the project complies with the terms of LGPL v3 and LGPL v2.1, SDKs can be created as open source. A commercial license is needed if the project is large enough, has a technical novelty, and the manufacturer needs to protect his interests. In other cases, an open license is sufficient.
• Highly efficient and smooth UI development: Qt is a time-tested and one of the best toolkits for creating multi-platform UIs. The framework has a minimum number of inconveniences and imperfections in comparison with other GUI platforms.
• The framework has incredible cross-platform capabilities: it supports desktop, mobile, and embedded platforms.
• Fast and easy prototyping: Qt allows engineers to turn designs into interactive prototypes very quickly. The design code is easily implemented into the program logic.
• Detailed documentation and a large community: The framework owners have created a huge database of documents, including tutorials, examples, case studies, and application development guides. Almost any question can be answered quickly and comprehensively by Qt users united in the community. Few frameworks can boast such support.
• Feedback: The Qt Company is in constant dialog with the users, improving the accessibility of the framework and making it convenient to work with. In February 2022, the company simplified and streamlined commercial licensing, making the framework even more business-friendly. Qt sets itself apart from other products in that it captures the user experience and puts it at the heart of product development, making the interaction between designers and developers and feedback loops more optimal.
• Qt 3D Studio offers a 3D UI development system that is in demand by both software developers and graphic designers. Qt designers have the opportunity to leverage an open architecture with access to the source code. The platform contains ready-made UI libraries and allows designers to create embedded systems with 3D or 2D user interfaces.

The Qt HMI framework supports the Java-script-based declarative QML language, built on top of Qt. QML enables great user experience design with rich graphical support. The language serves to define the visual GUI elements that are required to build and animate the visual interaction and the calling program.

QML allows designers and developers to create, try, and change GUIs with simple JSON-based syntax and to transform a design into an HMI. UI elements written in QML can be integrated with logic written in both JavaScript and C++. QML helps specialists implement a project and test it on the target hardware without delay. That’s why HMI development using Qt provides significant advantages in terms of time to market.

There is no barrel of honey without a fly in the ointment. When considering using Qt to create HMI applications for industrial or home automation panels, the team must consider criteria for processor performance. The user interface of the Qt library needs about 100 DMIPS (Dhrystone MIPS). The selected microprocessor must have sufficient computing power to drive a screen with good resolution characteristics and process graphics quickly.

Example: Qt for automotive HMI development

The automotive industry has long been heading for consolidation, which means that the automotive software industry should adjust to such changes. HMI tools need to support cross-platform capabilities and be consistent with multi-brand variant management. Qt is successfully used to create automotive HMI solutions. In the automotive industry, hardware components are constantly being upgraded for more power. The development of the HMI UI should keep pace with this process. Qt simplifies the process of creating HMIs for the development team through the ease and speed of creating applications with a rich graphical interface, internationalization, and cross-platform support.

Such legendary automobile companies as Mercedes-Benz, Hyundai, Citroen, DS, Opel, and Peugeot were announced to have in-vehicle HMIs created with the help of the Qt framework. 

Thus, Qt is Hyundai Motor Group's HMI technology partner for all Hyundai, Kia, and Genesis vehicles. Qt has been selected to develop Hyundai's connected automotive operating system for the in-car infotainment system.

The system monitors and manages key aspects of connected automotive infrastructure to ensure that vehicles can adapt to innovations in safety, comfort, and convenience. Using a single code base, Qt enables the automotive company to create complex technology solutions to meet requirements in various areas, such as in-vehicle infotainment (IVI) and cluster/HUD. Qt Design Studio is claimed to be used by Hyundai Motor Group to speed up its HMI development process from prototype to deployment.

The role of software in the automotive industry will only increase. By offering state-of-the-art cross-platform solutions that are scalable across multiple variants, Qt HMI development is moving into a new phase, becoming the de-facto leader in its niche.

The Qt software development platform is used by companies all over the world, including well-known giants such as LG Electronics, Bosch, and Rockwell Automation. Regardless of the size of the enterprise, Qt enables companies to keep pace with the changing software market. Using Qt gives companies the ability to unify their HMI development process to greatly improve development efficiency, speed up time to market, and meet consumer expectations.

Qt tools for building embedded HMI software

The Qt development service offers Qt widgets and Qt Quick for UI creation. Qt Widgets have a lot of elements that indirectly depend on the platform on which the application is running. Qt Quick, in turn, is devoid of this shortcoming. Qt Quick is a library that allows you to create UIs using the QML language. QML provides a very convenient way to design UIs for touch displays. It comes out of the box with support for gestures, while Qt Widgets doesn't. Qt Quick has made it possible to quickly develop applications for mobile devices while logic can still be written using native code to achieve the highest possible performance.

With the Qt SDK, APIs for media, web, navigation, and more are available to application developers. These APIs serve to make HMI development easier and faster. The Qt SDK provides the Qt software stack and libraries for developing applications for your hardware. The Qt SDK combines Qt software components and all Qt modules, providing a smooth design process for custom embedded applications. Needed Qt modules should be chosen to create a customized Qt SDK and plug them into Qt Creator, a cross-platform IDE. Qt Creator automatically identifies Qt SDKs. The Qt SDK offers an easy setup with a single installation package to get started. By using the simulator, developers can get an idea of how their application will function on Qt-enabled devices. In this case, there is no need to use emulator devices. The Qt Designer, one of the Qt Creator tools, is used to create an interface design layout. The C++ code generated by Qt Designer is integrated into the application.

Qt HMI Development: Use Cases and Challenges

As a passionate software and hardware development company, Integra Sources uses the latest approaches and solutions that help to get the result with the least amount of time and money for the client. As a Qt development company, we have gained enough experience to share our opinion about the capabilities of this framework, including in the field of HMI application development.

Using the rich features of C++ and Qt, we create intuitive UIs with a variety of graphics, widgets, and visual effects for embedded system applications, desktop, console, and mobile applications. We are engaged in creating Qt software for embedded systems, including complex automation systems that support HMI.

We would like to cite our extensive work on implementing a battery management system (BMS) for a sophisticated industrial solution as an example of the Qt full-featured HMI project.

Having started a major project to develop BMS firmware and embedded software to communicate with it, one of the tasks facing our team was to develop a full-screen application for a color touch panel with a 1920x1080 screen resolution to display the status of the BMS.

BMS has a multi-level structure, consisting of cells (batteries) combined into modules. The modules are, in turn, assembled into strings.

Screens (main screen, line screen, and module screen) are provided for each level to display related information. There are also screens for displaying system alarms, system management options (control screen), firmware versions, and system components (About screen). We also needed to delineate the rights of system users as guest, service, and super.

The second task was to implement a touchscreen application for testing the finished BMS module. The application must run on a specially prepared computer. The test results must be loaded into a separate JSON file. Limit values ​​​​(e.g., maximum and minimum cell voltage, etc.) are set on the settings screen.

By using Qt Creator and working with C++ and QML, our Qt developers have created intuitive HMI applications with customized interfaces. HMI applications were written from scratch. By using detailed documentation provided by the Qt framework, necessary components (buttons, drop boxes, input fields, checkboxes, etc.) were set up fairly easily by developers, while additional buttons and other missing components were created from scratch. The help of a large community allowed developers to solve challenges that arose during the development of applications. For example, in a long list of parameters that can be edited and saved, when scrolling, the newly entered data is reset to the old data that was stored in the C++ model. The configuration of the widget caching (ListView) enabled the solution to the problem.

Like any universal tool, Qt has its weak points compared to narrowly focused native instruments.

Although Qt provides flexible tools for creating animations in a few lines of code, there are certain challenges in developing a GUI in Qt. Elaborate animation that is constantly present on the screen can consume a lot of processor resources. Speaking of the Raspberry Pi, consumption can reach 100%. The HMI screen will simply overheat.

The Qt animation optimization is based on the OpenGL library and does not involve optimization of shaders since not all video cards will support them. To create animations without shaders, Qt has to use many more resources.

The Qt framework provides the logic of behavior and interaction, but it doesn't give you a ready-made solution for visual display. This approach has a certain complexity since it requires a lot of experience and extensive knowledge from the programmer. At the same time, Qt gives you a flexible way to choose styles and options, saving you from extensive rework within the system.

In one of our projects, the implementation of the HMI part included, among other things, creating a table with a non-standard visual design and functionality.

For creating tables, the Qt framework offers only the components that implement functionality at the simplest level - just an array of text with a standard font, a set of columns, and rows of a fixed size without alignment. For non-standard designs, you must manually fill in the elements of the table.

The client's wish was to implement more sophisticated functionality than the base Qt implementation for greater clarity and ease of use of the table.

To build a table with the given parameters, we have implemented useful classes:

• You can use controller classes that allow you to display a table in the desired form. A controller class was implemented to build a table in its style. The Qt component can use this model to get table commands and configuration.
• data filtering with the ability to sort in real-time
• column and row parameters, as well as their participation in table sorting and searching;
• The controller classes allow the table to adapt to the HMI screen.

Qt's great advantage is that it allows you to write user-friendly code that can be applied anywhere. The table as a ready-made component is easily scalable and can be used for other projects with minor modifications.

Conclusion

Human-machine interfaces now range from basic panels for a visual representation of the state of the control system to complex environments with multimedia and networking capabilities. HMI is increasingly being applied in consumer and industrial applications. Ease of use, efficiency, and a wide range of connectivity options are the driving forces behind the massive change in HMI these days. As embedded devices become smaller, designers have to solve complicated tasks. It is necessary to include many useful functions in a miniature product to ensure efficient operation. The time to market for devices is also being shortened to ensure their competitiveness and business profitability. Various GUI toolkits can help create HMI software with some success and comfort for designers.

Long and successful experience in using Qt enables Integra Sources to consider this framework to be the most convenient set of tools for HMI software development. We have applied the Qt framework to create fascinating, multifunctional, and user-friendly human-machine interfaces of different difficulty levels. Our expert assistance in developing embedded apps is at your disposal.