Detection of the farmland plow areas using RGB-D images with an improved YOLOv5 model

Jiangtao Ji, Zhihao Han, Kaixuan Zhao, Qianwen Li, Shucan Du

Abstract


Recognition of the boundaries of farmland plow areas has an important guiding role in the operation of intelligent agricultural equipment. To precisely recognize these boundaries, a detection method for unmanned tractor plow areas based on RGB-Depth (RGB-D) cameras was proposed, and the feasibility of the detection method was analyzed. This method applied advanced computer vision technology to the field of agricultural automation. Adopting and improving the YOLOv5-seg object segmentation algorithm, first, the Convolutional Block Attention Module (CBAM) was integrated into Concentrated-Comprehensive Convolution Block (C3) to form C3CBAM, thereby enhancing the ability of the network to extract features from plow areas. The GhostConv module was also utilized to reduce parameter and computational complexity. Second, using the depth image information provided by the RGB-D camera combined with the results recognized by the YOLOv5-seg model, the mask image was processed to extract contour boundaries, align the contours with the depth map, and obtain the boundary distance information of the plowed area. Last, based on farmland information, the calculated average boundary distance was corrected, further improving the accuracy of the distance measurements. The experiment results showed that the YOLOv5-seg object segmentation algorithm achieved a recognition accuracy of 99% for plowed areas and that the ranging accuracy improved with decreasing detection distance. The ranging error at 5.5 m was approximately 0.056 m, and the average detection time per frame is 29 ms, which can meet the real-time operational requirements. The results of this study can provide precise guarantees for the autonomous operation of unmanned plowing units.
Keywords: plow areas, RGB-D camera, YOLO, object segmentation, contour boundary, average distance
DOI: 10.25165/j.ijabe.20241703.8810

Citation: Ji J T, Han Z H, Zhao K X, Li Q W, Du S C. Detection of the farmland plow areas using RGB-D images with an improved YOLOv5 model. Int J Agric & Biol Eng, 2024; 17(4): 156-165.

Keywords


plow areas, RGB-D camera, YOLO, object segmentation, contour boundary, average distance

Full Text:

PDF

References


Luo X W, Liao J, Zang Y, Qu Y G, Wang P. Developing from mechanized to smart agricultural production in China. Strategic Studi of CAE, 2022; 24（1）: 46–54.

Liu C L, Lin H Z, Li Y M, Gong L, Miao Z H. Analysis on status and development trend of intelligent control technology for agricultural equipment. Transactions of the CSAM, 2020; 51（1）: 1–18.

Zhang M, Ji Y H, Li S C, Cao R Y, Xu H Z, Zhang Z Q. Research progress of agricultural machinery navigation technology. Transactions of the CSAM, 2020; 51（4）: 1–18. （in Chinese）

Zhou J, Ji C Y, Liu C L. Visual navigation system of agricultural wheeled-mobile robot. Transactions of the CSAM, 2005; 36（3）: 90–94. （in Chinese）

Chen B Q, Wu Z H, Li H Y, Wang J. Research of machine vision technology in agricultural application: Today and the future. Science & Technology Review, 2018; 36（11）: 54–65. （in Chinese）

Xie Y S, Chen K, Li W T, Zhang Y, Mo J Q. An improved adaptive threshold RANSAC method for medium tillage crop rows detection. In: 2021 6th International Conference on Intelligent Computing and Signal Processing （ICSP）, Xi’an: IEEE, 2021; pp.1282–1286.

Ahmadi A, Nardi L, Chebrolu N, Stachniss C. Visual servoing-based navigation for monitoring row-crop fields. In: 2020 IEEE International Conference on Robotics and Automation （ICRA）, Paris: IEEE, 2020; 4920–4926.

Zheng H, Wang Q, Ji J M. Navigation line extraction based on image processing for weeding robot. In: IECON 2022 - 48th Annual Conference of the IEEE Industrial Electronics Society, Brussels: IEEE, 2022; pp.1–5.

Xue J L, Zhang S S. Navigation of an agricultural robot based on laser radar. Transactions of the CSAM, 2014; 45（9）: 55–60. （in Chinese）

Yun C, Kim H-J, Jeon C-W, Kim J-H. Stereovision-based guidance line detection method for auto-guidance system on furrow irrigated fields. IFAC-PapersOnline, 2018; 51（17）: 157–161.

Zhao T, Noboru N, Yang L L, Kazunobu I, Chen J. Development of uncut crop edge detection system based on laser rangefinder for combine harvesters. Int J Agric & Biol Eng, 2016; 9（2）: 21–28.

Wang Q, Liu H, Yang P S, Meng Z J. Detection method of headland boundary line based on machine vision. Transactions of the CSAM, 2020; 51（5）: 18–27. （in Chinese）

Cai D Q, Li Y M, Tan C J, Liu C L. Detection method of boundary of paddy fields using support vector machine. Transactions of the CSAM, 2019; 50（6）: 22–27, 109. （in Chinese）

Zhu S X, Xie Z H, Xu H, Pan J D, Ren S G. An approach to field path detection based on computer vision. Jiangsu Journal of Agriculture, 2013; 29（6）: 1291–1296.

Ollis M, Stentz A. Vision-based perception for an automated harvester. In: Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems: Innovative Robotics for Real-world Applications. IROS'97, Grenoble: IEEE, 1997; 3: 1838–1844.

Åstrand B, Baerveldt A J. A vision based row-following system for agricultural field machinery. Mechatronics, 2005; 15（2）: 251–269.

Wei X H, Zhang M, Liu Q S, Li L. Extraction of crop height and cut-edge information based on binocular vision. Transactions of the CSAM, 2022; 53（3）: 225–233. （in Chinese）

Hong Z J, Li Y M, Lin H Z, Gong L, Liu C L. Field boundary distance detection method in early stage of planting based on binocular vision. Transactions of the CSAM, 2022; 53（5）: 27–33, 56. （in Chinese）

Chen B B, Cao Q X. Grain height and boundary detection based on based on laser and IMU. Mechatronics, 2020; 26（Z1）: 46–51.

Wei L G, Zhang X C, Wang F Z, Che Y, Sun X W, Wang Z W. Design and experiment of harvest boundary online recognition system for rice and wheat combine harvester based on laser detection. Transactions of the CSAE, 2017; 33（Z1）: 30–35. （in Chinese）

Zhang Y, Pan S Q, Xie Y S, Chen K, Mo J Q. Detection of ridge in front of vehicle based on fusion of camera and millimeter wave radar. Transactions of the CSAE, 2021; 37（15）: 169–178. （in Chinese）

Persello C, Tolpekin V A, Bergado J R, de By R A. Delineation of agricultural fields in smallholder farms from satellite images using fully convolutional networks and combinatorial grouping. Remote Sensing of Environment, 2019; 231: 111253.

Qiao Y J, Liu H, Meng Z J, Chen J P, Ma L Y. Method for the automatic recognition of cropland headland images based on deep learning. Int J Agric & Biol Eng, 2023; 16（2）: 216–224.

Li X Y, Cai C, Zhang R F, Ju L, He J R. Deep cascaded convolutional models for cattle pose estimation. Computers and Electronics in Agriculture, 2019; 164: 104885.

Wang C Y, Liao H Y M, Wu Y H, Chen P Y, Hsieh J W, Yeh I-H. CSPNet: A new backbone that can enhance learning capability of CNN. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle: 2020; pp.1571–1580.

Zhang Z, He T, Zhang H, Zhang Z Y, Xie J Y, Li M. Bag of freebies for training object detection neural networks. arXiv prepress, 2019.

Wang G, Fang H B, Wang D Z, Yan J W, Xie B L. Ceramic tile surface defect detection based on deep learning. Ceramics International, 2022; 48（8）: 11085–11093.

Syed T N, Liu J Z, Zhou X, Zhao S Y, Yuan Y, Ahmed M S H, Ali L I. Seedling-lump integrated non-destructive monitoring for automatic transplanting with Intel RealSense depth camera. Artificial Intelligence in Agriculture, 2019; 3: 18–32.

Xin Y X, Wang C Y. An image edge detection method based on Canny operator. Information & Computer, 2017; 18: 37–38, 41.

He K M, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. In: 2017 IEEE International Conference on Computer Visio （ICCV）, Venice: IEEE, 2017; pp.2980–2988.

Bolya D, Zhou C, Xiao F Y, Lee Y J. Yolact: Real-time Instance Segmentation. In: 2019 IEEE/CVF International Conference on Computer Vision （ICCV）, Seoul: IEEE, 2019; pp.9157–9166.




Copyright (c) 2024 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office