RGB-D visual saliency detection of stacked fruits under poor lighting

Chunjian Hua, Xintong Zou, Yi Jiang, Jianfeng Yu, Ying Chen

Abstract


The saliency detection of the same kind of stacked fruits can assist robots in completing sorting tasks, which is an important prerequisite for the grading and packing of fruits. In order to accurately obtain saliency targets of fruits in the same kind of stacked state under overexposure, non-uniform illumination, and low illumination, a method for detecting stacked fruits under poor illumination based on RGB-D visual saliency was proposed. Based on the Res2Net network, features from each layer of two images were obtained. To realize the complementary advantages between RGB features and depth features, the input RGB images were preprocessed using depth weighting to obtain purified RGB features. To increase the information interaction between branches of different scales and better balance the fusion features and modal exclusive features, a multiscale progressive fusion module was proposed. To minimize the difference between the initial saliency maps generated by different features and improve the accuracy of the final predicted saliency maps, a multi-branch hybrid supervised method was used. The comprehensive experiments on the self-made dataset of the same kind of stacked fruits show that the proposed algorithm is superior to five state-of-the-art RGB-D SOD methods in four key indicators: S value, F value, and MAE value, which are 0.979, 0.992, and 0.006, respectively, and the P-R curve, which is also closer to the upper right corner of the graph. These values demonstrate that the proposed algorithm can accurately obtain saliency targets in the same kind of stacked fruits. The results of this study can promote the automatic development of the fruit production and packaging industry.
Keywords: RGB-D salient object detection, multi-branch fusion, depth weighting, mixed supervision, same kind of stacked fruits
DOI: 10.25165/j.ijabe.20251801.8057

Citation: Hua C J, Zou X T, Jiang Y, Yu J F, Chen Y. RGB-D visual saliency detection of stacked fruits under poor lighting.
Int J Agric & Biol Eng, 2025; 18(1): 230–237.

Keywords


RGB-D salient object detection, multi-branch fusion, depth weighting, mixed supervision, same kind of stacked fruits

Full Text:

PDF

References


Dewi T, Risma P, Oktarina Y. Fruit sorting robot based on color and size for an agricultural product packaging system. Bulletin of Electrical Engineering and Informatics, 2020; 9(4): 1438–1445.

Fan S X, Liang X T, Huang W Q, Zhang V J L, Pang Q, He X, et al. Real-time defects detection for apple sorting using NIR cameras with pruning-based YOLOV4 network. Computers and Electronics in Agriculture, 2022; 193: 106715.

Kumaravel G, Ilankumaran V, Al Maqrashi S A A, Al Yaaqubi M K S. Automated date fruits sorting machine using fuzzy logic controller. International Journal of Recent Technology and Engineering, 2019; 8(4): 1089–1093.

Wang Y, Wang Z T. Salient object detection based on deep network. Electronic Measurement Technology, 2019; 42(21): 101–104. (in Chinese)

Tan D N, Liu Y, Yao L B, Ding Z R, Lu X Q. Semantic segmentation of multi-source remote sensing images based on visual attention mechanism. Journal of Signal Processing, 2022; 38(6): 1180–1191. (in Chinese)

Lyu P F, Wang Y, Wang S Q, Yu X S, Wu C D. Optic disc detection based on visual saliency in fundus image. Journal of Image and Graphics, 2021; 26(9): 2293–2304. (in Chinese)

Arivazhagan S, Shebiah R N, Harini R, Swetha S. Human action recognition from RGB-D data using complete local binary pattern. Cognitive Systems Research, 2019; 58: 94–104.

Liu D, Hu Y S, Zhang K, Chen Z Z. Two-stream refinement network for RGB-D saliency detection. In: 2019 IEEE International Conference on Image Processing (ICIP), Taipei: IEEE, 2019; pp.3925–3929.

Zhao J X, Cao Y, Fan D P, Cheng M M, Li X Y, Zhang L. Contrast prior and fluid pyramid integration for RGBD salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, 2020; pp.3922–3931. doi: 10.1109/CVPR.2019.00405.

Singh S K, Srivastava R. SL-Net: Self-learning and mutual attention-based distinguished window for RGBD complex salient object detection. Neural Computing and Applications, 2022; 35: 595–609.

Das D K, Shit S, Ray D N. Depth-guided two-way saliency network for 2D images. In: Advanced computational paradigms and hybrid intelligent computing. Springer, 2022; pp.61–71. doi: 10.1007/978-981-16-4369-9_7.

Ju R, Ge L, Geng W J, Ren T W, Wu G S. Depth saliency based on anisotropic center-surround difference. In: 2014 IEEE International Conference on Image Processing (ICIP), Paris: IEEE, 2014; pp.1115–1119. doi: 10.1109/ICIP.2014.7025222.

Peng H, Li B, Xiong W H, Hu W M, Ji R R. RGBD salient object detection: A benchmark and algorithms. In: Proceedings of the European Conference on Computer Vision (ECCV), 2014; pp.92–109. doi: 10.1007/978-3-319-10578-9_7.

Gao S H, Cheng M M, Zhao K, Zhang X Y, Yang M H, Torr P. Res2Net: A new multi-scale backbone architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019; 43(2): 652–662.

Zhang W B, Ji G P, Wang Z, Fu K R, Zhao Q J. Depth quality-inspired feature manipulation for efficient RGB-D salient object detection. In: Proceedings of the 29th ACM International Conference on Multimedia, 2021; pp.731–740. doi: 10.1145/3474085.345240.

Zhang P P, Wang D, Lu H C, Wang H Y, Ruan X. Amulet: Aggregating multi-level convolutional features for salient object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice: IEEE, 2017; pp.202–211. doi: 10.1109/ICCV.2017.31.

Ji W, Li J J, Yu S, Zhang M, Piao Y R, Yao S Y. Calibrated RGB-D salient object detection. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville: IEEE, 2021; pp.9466–9476. doi: 10.1109/CVPR46437.2021.00935.

Wu Z, Su L, Huang Q M. Cascaded partial decoder for fast and accurate salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach: IEEE, 2019; pp.3902–3911. doi: 10.1109/CVPR.2019.00403.

Zhou T, Fu H Z, Chen G, Zhou Y, Fan D P, Shao L. Specificity-preserving RGB-D saliency detection. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal: IEEE, 2021; pp.4661–4671. doi: 10.1109/ICCV48922.2021.00464.

Qin X B, Zhang Z C, Huang C Y, Gao C, Dehghan M, Jagersand M. BASNet: Boundary-aware salient object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach: IEEE, 2019; pp.7471–7481. doi: 10.1109/CVPR.2019.00766.

Steiner B, Devito Z, et al. PyTorch: An imperative style, high performance deep learning library. arXiv, 2019; arXiv: 1912.01703. In Press. doi: 10.48550/arXiv.1912.01703.

Kingma D, Ba J. Adam: A method for stochastic optimization. arXiv, 2014; arXiv: 1412.6980. doi: 10.48550/arXiv.1412.6980.

Achanta R, Hemami S, Estrada F, Susstrunk S. Frequency-tuned salient region detection. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami: IEEE, 2009; pp.1597–1604. doi: 10.1109/CVPR.2009.5206596.

Borji A, Cheng M M, Jiang H Z, Li J. Salient object detection: A benchmark. IEEE Transactions on Image Processing, 2015; 24(12): 5706–5722.

Fan D P, Cheng M M, Liu Y, Li T, Borji A. Structure-measure: A new way to evaluate foreground maps. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice: IEEE, 2017; pp.4558–4567. doi: 10.1109/ICCV.2017.487.

Fan D P, Lin Z, Zhang Z, Zhu M L, Cheng M M. Rethinking RGB-D salient object detection: Models, data sets, and large-scale benchmarks. IEEE Transactions on Neural Networks and Learning Systems, 2020; 32(5): 2075–2089.

Piao Y J, Ji W, Li J J, Zhang M, Lu H C. Depth-induced multi-scale recurrent attention network for saliency detection. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul: IEEE, 2020; pp.7253–7262. doi: 10.1109/ICCV.2019.00735.

Piao Y R, Rong Z K, Zhang M, Ren W S, Lu H C. A2dele: Adaptive and attentive depth distiller for efficient RGB-D salient object detection. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, 2020; pp.9057–9066. doi: 10.1109/CVPR42600.2020.00908.

Chen T Y, Hu X G, Xiao J, Zhang G F, Wang S J. CFIDNet: Cascaded feature interaction decoder for RGB-D salient object detection. Neural Computing and Applications, 2022; 34(10): 7547–7563.

Cheng Y P, Fu H Z, Wei X X, Xiao J J, Cao X C. Depth enhanced saliency detection method. In: Proceedings of International Conference on Internet Multimedia Computing and Service, 2014; pp.23–27. doi: 10.1145/2632856.2632866.




Copyright (c) 2025 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office