This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
Virtual Camera Driver and Desktop Application for macOS
Custom software development
macOS driver development
Background
Video blogging has become a very popular activity over recent years. Some people shoot and publish videos as a hobby. Others make money by creating high-quality content. Creating professional video content requires a team and costly equipment. So, an Asian company came up with an idea of a device that could replace both.
They created a motion tracking device that works in tandem with any smartphone. The user attaches his or her phone to the device and launches the mobile application. The application uses the camera on the phone to take pictures and shoot videos. At the same time, the application controls the tracking device and its wide range of functions. Users can select cameras, change the tracking mode, choose the position of the figure in the frame, control brightness, and so on.
The movement tracking device can rotate 360° horizontally and 60° vertically. The mobile app uses artificial intelligence to recognize and follow objects by controlling the device. The face tracking mode makes the device keep a person’s face in the frame. The figure-follow mode allows users to select an object in the frame so that the camera follows it. The device has other functions as well, such as auto zoom, auto alignment, predictive follow, and more.
The device is relevant for video blogging, live broadcasting, and video conferences.
Request
The customer already had a completed device and a mobile application. But they needed a macOS desktop application that would receive the video and audio data from the mobile application, process it, and transmit it to a virtual camera. The virtual camera would then send the video and audio to other applications. Our job was to develop such an application.
Another task was to develop macOS video and audio drivers for the virtual camera. The drivers had to work with a wide range of applications and websites, such as Skype, Zoom, YouTube, and Facebook to name a few.
Solution
Our team had to complete three tasks: to create a macOS desktop application, as well as to design a video driver and an audio driver.
As for the desktop application, the team had already created its Windows version (as well as virtual webcam drivers) at the previous stage of our collaboration. It was written in C++ and JavaScript on the Electron framework. Since Electron is a cross-platform framework, porting the application to MaOS was an easy task. All we had to do is to change the driver interface part of the application to meet the new driver API.
The whole system was supposed to work as follows. The mobile application on a smartphone controls the movement tracking device and sends multimedia data (video and audio) to the desktop application via Wi-Fi. The desktop application demultiplexes the data into video and audio. After that, the data is sent to a virtual camera which is, in turn, connected to a video streaming application or website.
The team created the video driver by writing a DAL virtual camera plugin on the CoreMediaIO framework in Objective C and C++. Other video streaming applications can receive video data from this plugin which serves as a virtual camera.
The driver allows the desktop application to control the camera feed, namely brightness, contrast, color, video settings, and so on.
The audio driver was developed on the CoreAudio and IOKit frameworks in Objective C and C++.
The installation packages of the desktop application and the drivers for macOS were created in PKG.
Major Issues Resolved
One of the first problems the team faced was related to video data transmission from the desktop application to the DAL plugin. The video frames from the application required a special interface to be sent to the plugin. The transmission failed when dealing with large amounts of data. Initially, the team tried to create this interface with pasteboards. However, this attempt caused a lot of problems. So, we decided to use a socket connection instead.
Another major problem was synchronizing the video and audio data flows. We solved this issue algorithmically.
When developing the desktop application, we faced a demultiplexer issue: this function did not work properly. Also, sometimes videos did not fit the screen. But both these problems were solved when the team developed the Windows version of the application. At this phase, they did not need to be addressed.
The Scope of Work
- Video kernel extension (KEXT) development for macOS
- Audio kernel extension (KEXT) development for macOS
- Desktop application development
Technologies Used
- The desktop application was developed on the Electron framework.
- The application was implemented using C++ and TypeScript.
- We developed the video and audio KEXTs in Objective C and C++.
- The video KEXT was implemented using the CoreMediaIO framework.
- The audio KEXT was developed on the CoreAudio and IOKit frameworks.
Result
Integra Sources developed a desktop application for macOS, as well as video and audio drivers for the virtual camera so that the application could receive data from the mobile application and transmit it as a standard video to other applications and websites.
You might also like...
Industrial Emergency Detection System
The solution uses artificial intelligence to process images from IP cameras, detect dangerous incidents, and send an alarm to a Jmix application. 
LEARN MORE 
Licensing Service Development for 3D Printers
The service uses the SHA-256 hashing algorithm to verify the authenticity of licenses. We designed the solution, created a web application with Jmix and enhanced the functionality of the desktop application.
LEARN MORE