Segmentation of field grape bunches via an improved pyramid scene parsing network

Shan Chen, Yuyang Song, Jinya Su, Yulin Fang, Lei Shen, Zhiwen Mi, Baofeng Su

Abstract


With the continuous expansion of wine grape planting areas, the mechanization and intelligence of grape harvesting have gradually become the future development trend. In order to guide the picking robot to pick grapes more efficiently in the vineyard, this study proposed a grape bunches segmentation method based on Pyramid Scene Parsing Network (PSPNet) deep semantic segmentation network for different varieties of grapes in the natural field environments. To this end, the Convolutional Block Attention Module (CBAM) attention mechanism and the atrous convolution were first embedded in the backbone feature extraction network of the PSPNet model to improve the feature extraction capability. Meanwhile, the proposed model also improved the PSPNet semantic segmentation model by fusing multiple feature layers (with more contextual information) extracted by the backbone network. The improved PSPNet was compared against the original PSPNet on a newly collected grape image dataset, and it was shown that the improved PSPNet model had an Intersection-over-Union (IoU) and Pixel Accuracy (PA) of 87.42% and 95.73%, respectively, implying an improvement of 4.36% and 9.95% over the original PSPNet model. The improved PSPNet was also compared against the state-of-the-art DeepLab-V3+ and U-Net in terms of IoU, PA, computation efficiency and robustness, and showed promising performance. It is concluded that the improved PSPNet can quickly and accurately segment grape bunches of different varieties in the natural field environments, which provides a certain technical basis for intelligent harvesting by grape picking robots.
Keywords: grape bunches, semantic segmentation, deep learning, improved PSPNet
DOI: 10.25165/j.ijabe.20211406.6903

Citation: Chen S, Song Y Y, Su J Y, Fang Y L, Shen L, Mi Z W, et al. Segmentation of field grape bunches via an improved pyramid scene parsing network. Int J Agric & Biol Eng, 2021; 14(6): 185–194.

Keywords


grape bunches, semantic segmentation, deep learning, improved PSPNet

Full Text:

PDF

References


Yu Y, Zhang K, Yang L, Zhang D. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Computers and Electronics in Agriculture, 2019; 163: 104846. doi: 10.1016/ j.compag.2019.06.001.

Xiong J, Liu Z, Lin R, Bu R, He Z L, Yang Z G, Liang C. Green grape detection and picking-point calculation in a night-time natural environment using a charge-coupled device (CCD) vision sensor with artificial illumination. Sensors (Basel, Switzerland), 2018; 18(4): 969. doi: 10.3390/s18040969.

Murillo-Bracamontes E A, Martinez-Rosas M E, Miranda-Velasco M M, Martinez-Reyes H L, Martinez-Sandoval J R, Cervantes-De-Avila H. Implementation of Hough transform for fruit image segmentation. Procedia Engineering, 2012; 35: 230–239. doi: 10.1016/j.proeng.2012.04. 185.

Reis M J, Morais R, Peres E, Pereira C, Contente O, Soares S, Valente A, Baptista J, Ferreira P J S, Cruz J B. Automatic detection of bunches of grapes in natural environment from color images. Journal of Applied Logic, 2012; 10(4): 285–290. doi: 10.1016/j.jal.2012.07.004.

Liu S, Whitty M. Automatic grape bunch detection in vineyards with an SVM classifier. Journal of Applied Logic, 2015; 13(4): 643–653. doi: 10.1016/j.jal.2015.06.001.

Pérez-Zavala R, Torres-Torriti M, Cheein F A, Troni G. A pattern recognition strategy for visual grape bunch detection in vineyards. Computers and Electronics in Agriculture, 2018; 151: 136–149. doi: 10.1016/j.compag.2018.05.019.

Cecotti H, Rivera A, Farhadloo M, Pedroza M A. Grape detection with convolutional neural networks. Expert Systems with Applications, 2020; 159: 113588. doi: 10.1016/j.eswa.2020.113588.

Milella A, Marani R, Petitti A, Reina G. In-field high throughput grapevine phenotyping with a consumer-grade depth camera. Computers and Electronics in Agriculture, 2019; 156: 293–306. doi: 10.1016/j.compag.2018.11.026.

Marani R, Milella A, Petitti A, Reina G. Deep neural networks for grape bunch segmentation in natural images from a consumer-grade camera. Precision Agriculture, 2021; 22(2): 387–413. doi: 10.1007/s11119-020- 09736-0.

Dias P A, Tabb A, Medeiros H. Multispecies fruit flower detection using a refined semantic segmentation network. IEEE robotics and automation letters, 2018; 3(4): 3003–3010.

Lin K, Gong L, Huang Y, Liu C, Pan J. Deep learning-based segmentation and quantification of cucumber powdery mildew using convolutional neural network. Frontiers in plant science, 2019; 10: 155. doi: 10.3389/fpls.2019.00155.

Ganchenko V, Starovoitov V, Zheng X. Image semantic segmentation based on high-resolution networks for monitoring agricultural vegetation. In: 2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), IEEE, 2020; pp.264–269. doi: 10.1109/SYNASC51798.2020.0050.

Tassis L M, De Souza J E T, Krohling R A. A deep learning approach combining instance and semantic segmentation to identify diseases and pests of coffee leaves from in-field images. Computers and Electronics in Agriculture, 2021; 186: 106191. doi: 10.1016/j.compag.2021.106191.

Chen S, Zhang K, Zhao Y, Sun Y, Ban W, Chen Y, et al. An approach for rice bacterial leaf streak disease segmentation and disease severity estimation. Agriculture, 2021; 11(5): 420. doi: 10.3390/ agriculture11050420.

Esgario J, Castro P, Tassis L M, Krohling R A. An app to assist farmers in the identification of diseases and pests of coffee leaves using deep learning. Information Processing in Agriculture, 2021; In press. doi: 10.1016/ j.inpa.2021.01.004.

Kang H, Chen C J S. Fruit detection and segmentation for apple harvesting using visual sensor in orchards. Sensors, 2019; 19(20): 4599. doi: 10.3390/s19204599.

Roy K, Chaudhuri S S, Pramanik S. Deep learning based real-time Industrial framework for rotten and fresh fruit detection using semantic segmentation. Microsystem Technologies, 2021; 27: 3365–3375.

Li J H, Tang Y C, Zou X J, Lin G C, Wang H J. Detection of fruit-bearing branches and localization of litchi clusters for vision-based harvesting robots. IEEE Access, 2020; 8: 117746–117758.

Kestur R, Meduri A, Narasipura O. MangoNet: A deep semantic segmentation architecture for a method to detect and count mangoes in an open orchard. Engineering Applications of Artificial Intelligence, 2019; 77: 59–69. doi: 10.1016/j.engappai.2018.09.011.

Shu B, Mu J, Zhu Y. AMNet: Convolutional neural network embeded with attention mechanism for semantic segmentation. In: Proceedings of the 2019 3rd High Performance Computing and Cluster Technologies Conference, 2019; pp.261–266. doi: 10.1145/3341069.3342988.

Lin C-Y, Chiu Y-C, Ng H F, Shih T K, Lin K-H. Global-and-local context network for semantic segmentation of street view images. Sensors, 2020; 20(10): 2907. doi: 10.3390/s20102907.

Wang Y, Lyu J, Xu L, Gu Y, Zou L, Ma Z. A segmentation method for waxberry image under orchard environment. Scientia Horticulturae, 2020; 266: 109309. doi: 10.1016/j.scienta.2020.109309.

Li Q, Jia W, Sun M, Hou S, Zheng Y. A novel green apple segmentation algorithm based on ensemble U-Net under complex orchard environment. Computers and Electronics in Agriculture, 2021; 180: 105900. doi: 10.1016/j.compag.2020.105900.

Amiri S A, Hassanpour H. A preprocessing approach for image analysis using gamma correction. International Journal of Computer Applications, 2012; 38(12): 38–46.

Woo S, Park J, Lee J-Y, Kweon I S. CBAM: Convolutional block attention module. In: Proceedings of the European conference on computer vision - ECCV 2018, Springer, Cham, 2018; pp.3–19. doi: 10.1007/978-3-030-01234-2_1.

Hesamian M H, Jia W, He X, Kennedy P J. Atrous convolution for binary semantic segmentation of lung nodule. In: ICASSP 2019 - 2019 IEEE international Conference on Acoustics, Speech and Signal Processing, IEEE, 2019; pp.1015–1019. doi: 10.1109/ICASSP.2019.8682220.

Fu H, Meng D, Li W, Wang Y. Bridge Crack Semantic Segmentation Based on Improved Deeplabv3+. Journal of Marine Science and Engineering, 2021; 9(6): 671. doi: 10.3390/jmse9060671.

Chen Y, Li Y, Wang J, Chen W, Zhang X. Remote sensing image ship detection under complex sea conditions based on deep semantic segmentation. Remote Sensing, 2020; 12(4): 625. doi: 10.3390/ rs12040625.

Wang W, Fu Y, Dong F, Li F. Semantic segmentation of remote sensing ship image via a convolutional neural networks model. IET Image Processing, 2019; 13(6): 1016–1022.

Huang L, He M, Tan C, Jiang D, Li G, Yu H. Jointly network image processing: multi-task image semantic segmentation of indoor scene based on CNN. IET Image Processing, 2020; 14(15): 3689–3697. doi: 10.1049/iet-ipr.2020.0088.

Pi Y, Nath N D, Behzadan A H. Detection and semantic segmentation of disaster damage in UAV footage. Journal of Computing in Civil Engineering, 2021; 35(2): 04020063. doi: 10.1061/(asce)cp.1943-5487. 0000947.




Copyright (c) 2021 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office