1. Home
  2. /
  3. Projects
  4. /
  5. Agricultural Robot with Computer Vision Algorithms
Integra Sources home button.

GET IN TOUCH

Agricultural Robot with Computer Vision Algorithms

Agriculture Computer vision Embedded software development

Backstory

Since herbal remedies and cosmetics are extremely popular nowadays, growing medicinal herbs could be a rather profitable business. Our customer, a farmer, decided to grow and harvest precious herbs and make tinctures and cosmetic products based on them.

The plantations of curative herbs are in a pristine area far from settlements and roads. It turned out to be a tough task for a farmer to find responsible employees, create a comfortable working environment, and arrange the delivery of the staff. That's why he started thinking about the step-by-step automation of gardening. 

The farmer intended to use an expensive solution for feeding plants with precise dosing. That was another point in favor of automation because human mistakes would cost too much.

An agricultural robot for feeding the plants.

Request

A farmer came to us with the idea of an agricultural robot for feeding plants on the beds.

A wheeled robot should be equipped with a manipulator arm. The device must drive along the bed, determine the location of the planting hole (or a plant), direct the manipulator arm to it, and spray the solution. The device should then rotate the arm and repeat the same actions with the hole in the next row. Having reached the edge of the planting bed, the device moves to another one.

The robot must work autonomously because the plot is located outside a stable Internet connection and mobile communication range.

The customer provided us with a moving base with an electric motor and controllers. We had to choose a ready-made manipulator, assemble the robot, and write software to control it.

The farmer has a plot with raised beds, rectangular in shape, covered with landscape fabric with planting holes in it. The width of the robot matches the bed, and the height is sufficient to hover above the plants.

Landscape fabric, which covers a planting bed, provides greater irrigation efficiency, prevents weeds from growing, and allows the soil to heat up evenly.

The device moves at a constant low speed along the bed and rotates a 3-axis desk robotic arm, applying the solution to each hole or the plant if the bush is large enough and the hole is not visible. A special sign placed at the end of each row tells the robot to turn to the next plant bed.

The agricultural robot moves along the planting beds, turning the arm to one or another side to fertilize the plantings.
Moving along the beds, the robot processes two rows of plantings one by one.

The robot moves autonomously and gives a light signal when it stops. It carries a reservoir with a nutrient solution on top. The amount of solution should be enough to treat the entire field. However, a liquid level sensor placed in the reservoir sends a signal to the light bulb to go on if the level drops to the minimum, and then the robot stops moving.

The robot is not afraid of even heavy rain and mud, as all the electronic components are protected from moisture.

We applied the OpenCV library and the Jetson Nano toolkit to develop computer vision algorithms that allow the robot to identify a planting hole, turn the manipulator arm in its direction, and spray the required amount of solution. Everything works in real time, and the robot does not need to stop by each planting hole. The device uses an Intel RealSense depth camera, which determines the distance to an object.

Integra Sources specialists prepared photo and video materials for teaching the system to recognize plant holes in a bed covered with landscape fabric.
Training the system to recognize planting holes.

Major Issues Resolved

A project involving computer vision software development is usually not easy to implement.

It took us some time to figure out all the ins and outs of a very specific Jetson Nano microcomputer. Our specialists prepared a data set and manually collected the video materials needed to teach the system to recognize given objects. There were some difficulties related to the picture contrast: the holes with soil were poorly recognized on black landscape fabric. Eventually, we managed to conquer all the challenges.

The planting holes in the ground merge with the black landscape fabric and are poorly recognized by computer vision algorithms.

The Scope of Work

  • Computer vision algorithm development;
  • Embedded software development;
  • Robot assembling and testing.

Technologies Used

  • We used the Python language and the OpenCV library to develop the algorithms.
  • The device is powered by the NVIDIA Jetson Nano module.
  • We used the Jetson Nano Toolkit to create computer vision algorithms.
  • The embedded software is written in C++ and runs on ROS.
  • We used CoppeliaSim as a simulation modeling environment that has ROS integration.
  • The robot is equipped with an Intel RealSense depth camera.

Result

We managed to create an agricultural robot that perfectly copes with its tasks on the farmer’s plot. The device has an electric motor powered by a gasoline generator, which is more convenient in the field than battery power.

The device works autonomously, fertilizing the beds one after another. The device turns on a light signal and stops in case it has detected an obstacle, processed all plants, or the solution level in the tank is low.

The farmer has positively assessed the economic effect of using our device and is now making plans for future automation of farming processes and, therefore, our cooperation.