Automatic sweet pepper detection based on point cloud images using subtractive clustering

Xiaokang Zhao, Hao Li, Qibing Zhu, Min Huang, Ya Guo, Jianwei Qin

Abstract


Automatic identification and detection of fruit on trees by machine vision is the basis of developing automatic harvesting robots in agriculture. The occlusion of branches, leaves and other fruits in canopy images will affect the accuracy of fruit detection. To provide a scientific and reliable technical guidance for fruit harvesting robots, a method using point cloud images was proposed in this study to detect red fruits to overcome the impact of occlusion on detection. Firstly, the fruit regions were segmented from a tree’s point cloud by applying the color threshold of red and green. Then, the noise in fruit point clouds was removed with sparse outlier removal. Finally, the point cloud of each fruit was detected and counted based on the subtractive clustering algorithm. For the sweet pepper dataset, the true positive rate (TPR) is 90.69% and the false positive rate (FPR) is 6.97% for all fruits that are at least partially visible in the scene.
Keywords: sweet pepper detection, point cloud, subtractive clustering, computer vision
DOI: 10.25165/j.ijabe.20201303.5460

Citation: Zhao X K, Li H, Zhu Q B, Huang M, Guo Y, Qin J W. Automatic sweet pepper detection based on point cloud images using subtractive clustering. Int J Agric & Biol Eng, 2020; 13(3): 154–160.

Keywords


sweet pepper detection, point cloud, subtractive clustering, computer vision

Full Text:

PDF

References


Shamshiri R R, Hameed I A, Karkee M, Weltzien C. Robotic harvesting of fruiting vegetables: A simulation approach in V-REP, ROS and MATLAB. In: Kulshreshtha S N (Ed.). Automation in Agriculture-Securing Food Supplies for Future Generations, London: InTechOpen, 2018; pp.81–105.

Shamshiri R R, Weltzien C, Hameed I A, Yule I J, Grift T E, Balasundram S K, et al. Research and development in agricultural robotics: A perspective of digital farming. Int J Agric & Biol Eng, 2018; 11(4): 1–14.

Gongal A, Amatya S, Karkee M, Zhang Q, Lewis K. Sensors and systems for fruit detection and localization: A review. Computers and Electronics in Agriculture, 2015; 116: 8–19.

Edan Y, Rogozin D, Flash T, Miles G E. Robotic melon harvesting. IEEE Transactions on Robotics and Automation, 2000; 16(6): 831–835.

Lu Q, Cai J R, Liu B, Deng L, Zhang Y J. Identification of fruit and branch in natural scenes for citrus harvesting robot using machine vision and support vector machine. Int J Agric & Biol Eng, 2014; 7(2): 115–121.

Tabb A L, Peterson D L, Parker J. Segmentation of Apple fruit from video via background modeling. In: ASABE Annual International Meeting Presentation. St. Joseph: ASABE, 2006; pp.60–63.

Kurtulmus F, Lee W S, Vardar A. Immature peach detection in colour images acquired in natural illumination conditions using statistical classifiers and neural network. Precision Agriculture, 2014; 15: 57–79.

Khoshroo A, Arefi A, Khodaei J. Detection of red tomato on plants using image processing techniques. Agricultural Communications, 2014; 2(4): 9–15.

Kuang H L, Liu C R, Chan L L H, Yan H. Multi-class fruit detection based on image region selection and improved object proposals. Neurocomputing, 2018; 283: 241–255.

Sa I, Ge Z Y, Dayoub F, Upcroft B, Perez T, McCool C. Deepfruits: A fruit detection system using deep neural networks. Sensors, 2016; 16(8): 1222.

Shamshiri R R, Hameed I A, Pitonakova L, Weltzien C, Balasundram S K, Yule I J. Simulation software and virtual environments for acceleration of agricultural robotics: Features highlights and performance comparison. Int J Agric & Biol Eng, 2018; 11(4): 15–31.

Wang Q, Nuske S, Bergerman M, Singh S. Automated crop yield estimation for apple orchards. Experimental Robotics, 2013; 88: 745-758.

Xiang R, Jiang H, Ying Y. Recognition of clustered tomatoes based on binocular stereo vision. Computers and Electronics in Agriculture, 2014; 106: 75–90.

Wei X, He J C, Ye D P, Jie D F. Navel orange maturity classification by multispectral indexes based on hyperspectral diffuse transmittance imaging. Journal of Food Quality, 2017; 7: 1–7.

Underwood J P, Hung C, Whelan B, Sukkarieh S. Mapping almond orchard canopy volume, flowers, fruit and yield using lidar and vision sensors. Computers and Electronics in Agriculture, 2016; 130: 83–96.

Stein M, Bargoti S, Underwood J. Image based mango fruit detection, localisation and yield estimation using multiple view geometry. Sensors, 2016; 16(11): 1915.

Gan H, Lee W S, Alchanatis V, Ehsani R, Schueller J K. Immature green citrus fruit detection using color and thermal images. Computers and Electronics in Agriculture, 2018; 152: 117–125.

Gongal A, Silwal A, Amatya S, Karkee M, Zhang Q, Lewis K. Apple crop-load estimation with over-the-row machine vision system. Computers and Electronics in Agriculture, 2016; 120: 26–35.

Tao Y T, Zhou J. Automatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking. Computers and Electronics in Agriculture, 2017; 142(A): 388–396.

Qureshi W S, Payne A, Walsh K B, Linker R, Cohen O, Dailey M N. Machine vision for counting fruit on mango tree canopies. Precision Agriculture, 2017; 18: 224–244.

Sa I, Lehnert C, English A, Mccool C, Dayoub F, Upcroft B, et al. Peduncle detection of sweet pepper for autonomous crop harvesting— combined color and 3-D information. IEEE Robotics and Automation Letters, 2017; 2(2): 765–772.

Lehnert C, Sa I, McCool C, Upcroft B, Perez T. Sweet pepper pose detection and grasping for automated crop harvesting. In: 2016 IEEE International Conference on Robotics and Automation (ICRA). Stockolm: IEEE, 2016; pp.2428–2434.

Rusu R B, Cousins S. 3D is here: Point cloud library (PCL). In: 2011 IEEE International Conference on Robotics and Automation. Shanghai: IEEE, 2011; pp.1–4.

Rusu R B, Marton Z C, Blodow N, Dolha M, Beetz M. Towards 3d point cloud based object maps for household environments. Robotics and Autonomous Systems, 2008; 56(11): 927–941.

Fernandez D, Parra I, Sotelo M A, Revenga P, Gavilan M. 3D candidate selection method for pedestrian detection on non-planar roads. In: 2007 IEEE Intelligent Vehicles Symposium. Istanbul: IEEE, 2007; pp.1162–1167.

Wu G, Zhu Q B, Huang M, Guo Y, Qin J W. Automatic recognition of juicy peaches on trees based on 3D contour features and colour data. Biosystems Engineering, 2019; 188: 1–13.




Copyright (c) 2020 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office