This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
Agricultural Robot with Computer Vision Algorithms
Agriculture
Computer vision
Embedded software development
Backstory
Since herbal remedies and cosmetics are extremely popular nowadays, growing medicinal herbs could be a rather profitable business. Our customer, a farmer, decided to grow and harvest precious herbs and make tinctures and cosmetic products based on them.
The plantations of curative herbs are in a pristine area far from settlements and roads. It turned out to be a tough task for a farmer to find responsible employees, create a comfortable working environment, and arrange the delivery of the staff. That's why he started thinking about the step-by-step automation of gardening.
The farmer intended to use an expensive solution for feeding plants with precise dosing. That was another point in favor of automation because human mistakes would cost too much.
Request
A farmer came to us with the idea of an agricultural robot for feeding plants on the beds.
A wheeled robot should be equipped with a manipulator arm. The device must drive along the bed, determine the location of the planting hole (or a plant), direct the manipulator arm to it, and spray the solution. The device should then rotate the arm and repeat the same actions with the hole in the next row. Having reached the edge of the planting bed, the device moves to another one.
The robot must work autonomously because the plot is located outside a stable Internet connection and mobile communication range.
The customer provided us with a moving base with an electric motor and controllers. We had to choose a ready-made manipulator, assemble the robot, and write software to control it.
The farmer has a plot with raised beds, rectangular in shape, covered with landscape fabric with planting holes in it. The width of the robot matches the bed, and the height is sufficient to hover above the plants.
The device moves at a constant low speed along the bed and rotates a 3-axis desk robotic arm, applying the solution to each hole or the plant if the bush is large enough and the hole is not visible. A special sign placed at the end of each row tells the robot to turn to the next plant bed.
The robot moves autonomously and gives a light signal when it stops. It carries a reservoir with a nutrient solution on top. The amount of solution should be enough to treat the entire field. However, a liquid level sensor placed in the reservoir sends a signal to the light bulb to go on if the level drops to the minimum, and then the robot stops moving.
The robot is not afraid of even heavy rain and mud, as all the electronic components are protected from moisture.
We applied the OpenCV library and the Jetson Nano toolkit to develop computer vision algorithms that allow the robot to identify a planting hole, turn the manipulator arm in its direction, and spray the required amount of solution. Everything works in real time, and the robot does not need to stop by each planting hole. The device uses an Intel RealSense depth camera, which determines the distance to an object.
Major Issues Resolved
A project involving computer vision software development is usually not easy to implement.
It took us some time to figure out all the ins and outs of a very specific Jetson Nano microcomputer. Our specialists prepared a data set and manually collected the video materials needed to teach the system to recognize given objects. There were some difficulties related to the picture contrast: the holes with soil were poorly recognized on black landscape fabric. Eventually, we managed to conquer all the challenges.
The Scope of Work
- Computer vision algorithm development;
- Embedded software development;
- Robot assembling and testing.
Technologies Used
- We used the Python language and the OpenCV library to develop the algorithms.
- The device is powered by the NVIDIA Jetson Nano module.
- We used the Jetson Nano Toolkit to create computer vision algorithms.
- The embedded software is written in C++ and runs on ROS.
- We used CoppeliaSim as a simulation modeling environment that has ROS integration.
- The robot is equipped with an Intel RealSense depth camera.
Result
We managed to create an agricultural robot that perfectly copes with its tasks on the farmer’s plot. The device has an electric motor powered by a gasoline generator, which is more convenient in the field than battery power.
The device works autonomously, fertilizing the beds one after another. The device turns on a light signal and stops in case it has detected an obstacle, processed all plants, or the solution level in the tank is low.
The farmer has positively assessed the economic effect of using our device and is now making plans for future automation of farming processes and, therefore, our cooperation.
You might also like...
Licensing Service Development for 3D Printers
The service uses the SHA-256 hashing algorithm to verify the authenticity of licenses. We designed the solution, created a web application with Jmix and enhanced the functionality of the desktop application. 
LEARN MORE 
Multimedia Device with Remote Control: Audit and Firmware Improvement
The device is designed for stage performances. We improved its update mechanism by adding redirection processing and made minor changes to the firmware.
LEARN MORE