Nondestructive perception of potato quality in actual online production based on cross-modal technology

Qiquan Wei, Yurui Zheng, Zhaoqing Chen, Yun Huang, Changqing Chen, Zhenbo Wei, Shuiqin Zhou, Hongwei Sun, Fengnong Chen

Abstract


Nowadays, China stands as the global leader in terms of potato planting area and total potato production. The rapid and nondestructive detection of the potato quality before processing is of great significance in promoting rural revitalization and augmenting farmers’ income. However, existing potato quality sorting methods are primarily confined to theoretical research, and the market lacks an integrated intelligent detection system. Therefore, there is an urgent need for a post-harvest potato detection method adapted to the actual production needs. The study proposes a potato quality sorting method based on cross-modal technology. First, an industrial camera obtains image information for external quality detection. A model using the YOLOv5s algorithm to detect external green-skinned, germinated, rot and mechanical damage defects. VIS/NIR spectroscopy is used to obtain spectral information for internal quality detection. A convolutional neural network (CNN) algorithm is used to detect internal blackheart disease defects. The mean average precision (mAP) of the external detection model is 0.892 when intersection of union (IoU) = 0.5. The accuracy of the internal detection model is 98.2%. The real-time dynamic defect detection rate for the final online detection system is 91.3%, and the average detection time is 350 ms per potato. In contrast to samples collected in an ideal laboratory setting for analysis, the dynamic detection results of this study are more applicable based on a real-time online working environment. It also provides a valuable reference for the subsequent online quality testing of similar agricultural products.
Keywords: cross-modal technology, potato quality, YOLOv5s, VIS/NIR spectroscopy, online nondestructive detection
DOI: 10.25165/j.ijabe.20231606.8076

Citation: Wei Q Q, Zheng Y R, Chen Z Q, Huang Y, Chen C Q, Wei Z B, et al. Nondestructive perception of potato quality in actual online production based on cross-modal technology. Int J Agric & Biol Eng, 2023; 16(6): 280-290.

Keywords


cross-modal technology, potato quality, YOLOv5s, VIS/NIR spectroscopy, online nondestructive detection

Full Text:

PDF

References


Xie C H. Potato Industry: Status and Development. Journal of Huazhong Agricultural University (Social Sciences Edition), 2012; 97(1): 1–4. URL: http://hnxbl.cnjournals.net/hznydxsk/article/abstract/20120101?st=article_issue

Jing J, Li J W, Liao G P, Yu X J, Viray C. Methodology for Potatoes Defects Detection with Computer Vision. Proceedings. The 2009 International Symposium on Information Processing (ISIP 2009), 2009; 346. URL: https://www.researchgate.net/publication/242098282_Methodology_for_Potatoes_Defects_Detection_with_Computer_Vision

Ebrahimi E, Mollazade K, Arefi A. Detection of Greening in Potatoes using Image Processing Techniques. Journal of American Science, 2011; 7(3): 243–247. URL: https://www.researchgate.net/publication/243457611_Detection_of_Greening_in_Potatoes_using_Image_Processing_Techniques

Barnes M, Duckett T, Cielniak G, Stroud G, Harper G. Visual detection of blemishes in potatoes using minimalist boosted classifiers. Journal of Food Engineering, 2010; 98(3): 339–346. URL: https://www.mendeley.com/catalogue/

dd8-9d7c-3992-8fdf-2d9ce49f2853/

Elmasry G, Cubero S, Moltó E, Blasco J. l. In-line sorting of irregular potatoes by using automated computer-based machine vision system. Journal of Food Engineering, 2012; 112(1-2): 60-68. URL: https://www.sciencedirect.com/science/

article/abs/pii/S0260877412001690

Oppenheim D, Shani G. Potato Disease Classification Using Convolution Neural Networks. Advances in Animal Biosciences, 2017; 8(02): 244-249. URL: https://www.sciencedirect.com/science/article/abs/pii/S2040470017001376

Elsharif A A, Dheir I M, Mettleq A, Abu-Naser S S. Potato Classification Using Deep Learning. International Journal of Academic Pedagogical Research, 2020; 3(12): 1-8. URL: https://philpapers.org/rec/ELSPCU

Chen J D, Zhang D F, Nanehkaran Y A, Li D L. Detection of rice plant diseases based on deep transfer learning. Journal of the Science of Food and Agriculture, 2020; 100(7): 3246-3256. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/

jsfa.10365

Zhao G Y, Quan L X, Li H L, Feng H Q, Li S W, Zhang S H, et al. Real-time recognition system of soybean seed full-surface defects based on deep learning. Computers and Electronics in Agriculture, 2021; 187:106230. URL: https://www.sciencedirect.com/science/article/abs/pii/S0168169921002477

Ramos R P, Gomes J S, Prates R M, Filho E F, Teruel B J, Costa D S. Non‐invasive setup for grape maturation classification using deep learning. Journal of the Science of Food and Agriculture, 2021; 101(5): 2042-2051. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/jsfa.10824

Thuyet D Q, Kobayashi Y C, Matsuo M. A robot system equipped with deep convolutional neural network for autonomous grading and sorting of root-trimmed garlics. Computers and Electronics in Agriculture, 2020; 178:105727. URL: https://www.sciencedirect.com/science/article/abs/pii/S0168169920310140

Nouri-Ahmadabadi H, Omid M, Mohtasebi S S, Firouz M S. Design, Development and Evaluation of an Online Grading System for Peeled Pistachios Equipped with Machine Vision Technology and Support Vector Machine. Information Processing in Agriculture, 2017; 4(4): 333–341. URL: https://www.sciencedirect.com/

science/article/pii/S2214317316300117

Blasco J, Cubero S, Gómez-Sanchís J, Mira P, Moltó E. Development of a machine for the automatic sorting of pomegranate (Punica granatum) arils based on computer vision. Journal of Food Engineering, 2009; 90(1): 27-34. URL: https://www.sciencedirect.com/science/article/abs/pii/S0260877408002653

Hajjar G, Quellec S, Pépin J, Challois S, Joly G, Deleu C, et al. MRI investigation of internal defects in potato tubers with particular attention to rust spots induced by water stress. Postharvest Biology and Technology, 2021; 180:111600. URL: https://www.sciencedirect.com/science/article/abs/pii/S0925521421001393

Sosa P , Guild G , Burgos G , Bonierbale M, Felde T. Potential and application of X-ray fluorescence spectrometry to estimate iron and zinc concentration in potato tubers. Journal of Food Composition and Analysis, 2018; 70: 22-27. URL: https://www.sciencedirect.com/science/article/pii/S088915751830070X

López-Maestresalas A, Keresztes J C , Goodarzi M , Arazuri S, Jarén C, Saeys W. Non-destructive detection of blackspot in potatoes by Vis-NIR and SWIR hyperspectral imaging. Food Control, 2016; 70: 229-241. URL: https://www.sciencedirect.com/science/article/abs/pii/S0956713516303085

Wu L G, He J G, Liu G S, Wang S L, He X G. Detection of common defects on jujube using Vis-NIR and NIR hyperspectral imaging. Postharvest Biology and Technology, 2016; 112: 134-142. URL: https://www.sciencedirect.com/science/

article/abs/pii/S0925521415301083

Ye D D, Sun L J, Tan W Y, Che W K, Yang M C. Detecting and classifying minor bruised potato based on hyperspectral imaging. Chemometrics and Intelligent Laboratory Systems, 2018: 177: 129-139. URL: https://www.sciencedirect.com/

science/article/abs/pii/S0169743917306251

Jamshidi B , Minaei S , Mohajerani E , Ghassemian H. Reflectance Vis/NIR spectroscopy for nondestructive taste characterization of Valencia oranges. Computers and Electronics in Agriculture, 2012; 85: 64-69. URL: https://www.

sciencedirect.com/science/article/abs/pii/S0168169912000828

Peirs A, Lammertyn J, Ooms K, Nicolaı̈ B M. Prediction of the optimal picking date of different apple cultivars by means of VIS/NIR-spectroscopy. Postharvest Biology and Technology, 2001; 21(2): 189-199. URL: https://www.sciencedirect.

com/science/article/abs/pii/S0925521400001459

Tušek A J, Benković M, Malešić E, Marić L, Jurina T, Kljusurić J G. Rapid quantification of dissolved solids and bioactives in dried root vegetable extracts using near infrared spectroscopy. Spectrochimica acta Part A: Molecular and biomolecular spectroscopy, 2021; 261: 120074. URL: https://www.sciencedirect.

com/science/article/abs/pii/S138614252100651X

Moomkesh S , Mireei S A , Sadeghi M , Nazeri M. Early detection of freezing damage in sweet lemons using Vis/SWNIR spectroscopy. Biosystems Engineering, 2017; 164: 157-170. URL: https://www.sciencedirect.com/science/article/abs/

pii/S1537511017305007

Li J B, Huang W Q, Zhao C J, Zhang B H. A comparative study for the quantitative determination of soluble solids content, pH and firmness of pears by Vis/NIR spectroscopy. Journal of food engineering, 2013; 116(2): 324-332.

Scalisi A , O'Connell M G. Application of Visible/NIR spectroscopy for the estimation of soluble solids, dry matter and flesh firmness in stone fruits. Journal of the Science of Food and Agriculture, 2021; 101(5):2100-2107. URL: https://www.sciencedirect.com/science/article/abs/pii/S0260877412005596

Wang F, Li Y Y, Peng Y , Yang B N, Li L, Yin X Q. Hand-held Device for Non-destructive Detection of Potato Quality Parameters. Transactions of the Chinese Society for Agricultural Machinery, 2018; (49)7: 7. URL: https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2018&filename=NYJX201807042&uniplatform=NZKPT&v=Bhl832oeBgzPg99FnP-kZUbLtMML3R4SrSrF4CgEMvMDHTIFsyL_OYfdCcMM3L7W

Wang F, Li Y Y, Peng Y K, Yang B N, Li L, Liu Y C. Multi-Parameter Potato Quality Non-Destructive Rapid Detection by Visible/Near-Infrared Spectra. Spectroscopy and Spectral Analysis, 2018; 38(12): 3736. URL: https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=CJFDLAST2019&filename=GUAN201812016&uniplatform=NZKPT&v=yiHXcRaOJAICFkcD9KtCct5Emf1NoONfuDjHKdAkR5LfKG57CS5m0Iwfdl3jFTBI

Zhang X Y , Liu W , Xing L , et al. An Near-infrared Prediction Model for Quality Indexes of Potato Processing. infrared, 2012.

Zhu Z, Zeng S W, Li X Y, Zheng J. Nondestructive Detection of Blackheart in Potato by Visible/Near Infrared Transmittance Spectroscopy. Journal of Spectroscopy, 2015; 2015: 1-9. URL: https://kns.cnki.net/kcms/detail/

detail.aspx?dbcode=CJFD&dbname=CJFD2012&filename=HWAI201212009&uniplatform=NZKPT&v=P0yTJMNtpDOrTITF6rfUe2KOEie_0PcVOwdUhPUMDxa1w5nlgHpOsWlOYznyIAFm

Song Y, Wang X Z, Xie H L, Li L Q, Ning J M, Zhang Z Z. Quality Evaluation of Keemun black tea by fusing data obtained from near-infrared reflectance spectroscopy and computer vision sensors. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 2021; 252(5): 119522. URL: https://www.sciencedirect.com/science/article/abs/pii/S1386142521000986

Yu H L, Liang Y L, Liang H, Zhang Y Z. Recognition of wood surface defects with near infrared spectroscopy and machine vision. Journal of Forestry Research, 2019; 30(6): 2379-2386. URL: https://link.springer.com/article/10.1007/s11676-018-00874-w

Huang X Y, Xu H X, Wu L, Dai H, Yao L Y, Han F K. A data fusion detection method for fish freshness based on computer vision and near-infrared spectroscopy. Analytical Methods, 2016; 8(14): 2929-2935. URL: https://pubs.rsc.org/en/

content/articlelanding/2016/ay/c5ay03005f/unauth

Yin J F, Hameed S, Xie L J, Ying Y B. Non-destructive detection of foreign contaminants in toast bread with near infrared spectroscopy and computer vision techniques. Journal of Food Measurement and Characterization, 2021; 15(1): 189–198. URL: https://link.springer.com/article/10.1007/s11694-020-00627-6

Ministry of Agriculture of the PRC. Grades and specifications of potatoes, 2006.

Yafen Han, Chengxu Lu¨, Yanwei Yuan, Bingnan Yang, Zhao Qingliang, Cao Youfu, and Yin Xueqing. Pls-discriminant analysis on patato blackheart disease based on vis-nir transmission spectroscopy. Spectroscopy and Spectral Analysis, 41(4):1213, 2021. URL: https://kns.cnki.net/kcms/detail/detail.aspx?dbcode=CJFD&dbname=

CJFDLAST2021&filename=GUAN202104044&uniplatform=NZKPT&v=w9Z0y1AQjjAudCYEmiOnS8srLZdbyPiVv5HBjrn9LPTri15Zm5X57fHem8eJp4pZ

Jubayer F, Soeb J A, Mojumder A N, Paul M K, Barua P, Kayshar S, Akter S S, et al. Detection of mold on the food surface using yolov5. Current Research in Food Science, 2021; 4:724–728. URL: https://www.sciencedirect.com/

science/article/pii/S2665927121000812

Rong D, Wang H Y, Ying Y B, Zhang Z Y, Zhang Y S. Peach variety detection using VIS-NIR spectroscopy and deep learning. Computers and Electronics in Agriculture, 2020; 175: 105553. URL: https://www.sciencedirect.com/

science/article/abs/pii/S0168169920305354

Tazehkandi A A. Computer Vision with OpenCV 3 and Qt5: Build visually appealing, multithreaded, cross-platform computer vision applications. Packt Publishing Ltd, 2018.

Yao L J, Lu L, Zheng R. Study on detection method of external defects of potato image in visible light environment. 2017 10th International Conference on Intelligent Computation Technology and Automation (ICICTA), 2017; 118–122. URL: https://www.researchgate.net/publication/320829726_Study_on_

Detection_Method_of_External_Defects_of_Potato_Image_in_Visible_Light_Environment

Clark C J, McGlone V A, Jordan R B. Detection of Brownheart in ‘Braeburn’ apple by transmission NIR spectroscopy. Postharvest Biology and Technology, 2003: 28(1): 87-96. URL: https://www.sciencedirect.com/science/

article/abs/pii/S0925521402001229




Copyright (c) 2023 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office