Research advance in phenotype detection robots for agriculture and forestry
Abstract
Keywords: computer vision, plant phenotype detection robot, phenotyping analysis, sensor, evaluation system, device clustering
DOI: 10.25165/j.ijabe.20231601.7945
Citation: Wang Y Q, Fan J C, Yu S, Cai S Z, Guo X Y, Zhao C J. Research advance in phenotype detection robots for agriculture and forestry. Int J Agric & Biol Eng, 2023; 16(1): 14–25.
Keywords
Full Text:
PDFReferences
Wang Z, Xun Y, Wang Y K, Yang Q H. Review of smart robots for fruit and vegetable picking in agriculture. Int J Agric & Biol Eng, 2022; 15(1): 33–54.
Bechar A, Vigneault C. Agricultural robots for field operations: Concepts and components. Biosystems Engineering, 2016; 149: 94–111.
Itzhaky Y, Farjon G, Khoroshevsky F, Shpigler A, Bar-Hillel A. Leaf counting: Multiple scale regression and detection using deep CNNs. BMVC, Newcastle, 2018; 328p.
Dong M, Yu H, Zhang L, Wu M, Sun Z P, Zeng D, et al. Measurement method of plant phenotypic parameters based on image deep learning. In: Lakshmanna K, ed. Wireless Communications and Mobile Computing, 2022; 2022: 1–9.
He W, Beck M, Martin W. Vision intelligence and software control for the automation of plant micropropagation. Measurement Science and Technology, 1992; 3(1): 38. doi: 10.1088/0957-0233/3/1/005.
Zhang Y, Tian Z Z, Ma W Q, Zhang M, Yang L L. Hyperspectral detection of walnut protein contents based on improved whale optimized algorithm. Int J Agric & Biol Eng, 2022; 15(6): 235–241.
Chehab E W, Eich E, Braam J. Thigmomorphogenesis: A complex plant response to mechano-stimulation. Journal of Experimental Botany, 2009; 60: 43–56.
Guo C L, Tang Y N, Lu J S, Zhu Y, Cao W X, Cheng T, et al. Predicting wheat productivity: Integrating time series of vegetation indices into crop modeling via sequential assimilation. Agricultural and Forest Meteorology, 2019; 272–273: 69-80.
Zhou M K, Xia J F, Yang F, Zheng K, Hu M J, Li D, et al. Design and experiment of visual navigated UGV for orchard based on Hough matrix and RANSAC. Int J Agric & Biol Eng, 2021; 14(6): 176–184.
Taghavi Namin S, Esmaeilzadeh M, Najafi M, Brown T B, Borevitz J O. Deep phenotyping: Deep learning for temporal phenotype/genotype classification. Plant Methods, 2018; 14(1): 66. doi: 10.1186/s13007-018-0333-4.
Yang C. Plant leaf recognition by integrating shape and texture features. Pattern Recognition, 2021; 112: 107809. doi: 10.1016/j.patcog.2020.107809.
Liu W, Zou S S, Xu X L, Gu Q Y, He W Z, Huang J, et al. Development of UAV-based shot seeding device for rice planting. Int J Agric & Biol Eng, 2022; 15(6): 1–7.
Forrest K L, Bhave M. Major intrinsic proteins (MIPs) in plants: a complex gene family with major impacts on plant phenotype. Funct Integr Genomics, 2007; 7(4): 263–289.
Wang Y J, Mao Q Y, Zhu H Q, Zhang Y, Ji J M, Zhang Y Y. Multi-modal 3D object detection in autonomous driving: a survey. arXiv, 2021; 2106: 12735. doi: 10.48550/arXiv.2106.12735.
Jin X L, Li Z H, Atzberger C. Estimation of crop phenotyping traits using unmanned ground vehicle and unmanned aerial vehicle imagery. Remote Sensing, 2020; 12(6): 940. doi: 10.3390/rs12060940.
Schauer N, Semel Y, Roessner U, Gur A, Balbo I, Carrari F, et al. Comprehensive metabolic profiling and phenotyping of interspecific introgression lines for tomato improvement. Nature biotechnology, 2006; 24: 447–454.
Lu Y, Chen X Y, Wu Z X, Yu J Z, Wen L. A novel robotic visual perception framework for underwater operation. Frontiers of Information Technology & Electronic Engineering, 2022; 23: 1602–1619.
Puliti S, Dash J P, Watt M S, Breidenbach J, Pearse G D. A comparison of UAV laser scanning, photogrammetry and airborne laser scanning for precision inventory of small-forest properties. Forestry: An International Journal of Forest Research, 2020; 93(1): 150-162.
Yuan H L, Liu Y M, Song M H, Zhu Y, Cao W X, Jiang X P, et al. Design of the mechanical structure of a field-based crop phenotyping platform and tests of the platform. Agronomy, 2022; 12(9): 2162. doi: 10.3390/agronomy12092162.
Roldán J, Garcia-Aunon P, Garzón M, de León J, del Cerro J, Barrientos A. Heterogeneous multi-robot system for mapping environmental variables of greenhouses. Sensors, 2016; 16(7): 1018. doi: 10.3390/s16071018.
Huang M F, Xu G Q, Li J Y, Huang J P. A method for segmenting disease lesions of maize leaves in real time using attention YOLACT++. Agriculture, 2021; 11(12): 1216. doi: 10.3390/agriculture11121216.
Chen G, Muriki H, Pradalier C, Chen Y S, Dellaert F. A hybrid cabledriven robot for non-destructive leafy plant monitoring and mass estimation using structure from motion. arXiv, 2022; 2209: 08690. doi: 10.48550/arXiv.2209.08690.
Jiang Z, Guo Y, Jiang K, Hu M, Zhu Z. Optimization of intelligent plant cultivation robot system in object detection. IEEE Sensors Journal, 2021; 21: 19279–19288.
Xie Z J, Gu S, Chu Q, Li B, Fan K J, Yang Y L, et al. Development of a high-productivity grafting robot for Solanaceae. Int J Agric & Biol Eng, 2020; 13(1): 82–90.
Young S N, Kayacan E, Peschel J M. Design and field evaluation of a ground robot for high-throughput phenotyping of energy sorghum. Precision Agric, 2019; 20(4): 697–722.
Atefi A, Ge Y, Pitla S, Schnable J. Robotic technologies for high-throughput plant phenotyping: Contemporary reviews and future perspectives. Front Plant Sci, 2021; 12: 611940. doi: 10.3389/FPLS.2021.611940.
Paraforos D S, Reutemann M, Sharipov G, Werner R, Griepentrog H W. Total station data assessment using an industrial robotic arm for dynamic 3D in-field positioning with sub-centimetre accuracy. Computers and Electronics in Agriculture, 2017; 136: 166–175.
Potena C, Nardi D, Pretto A. Fast and accurate crop and weed identification with summarized train sets for precision agriculture. Intelligent Autonomous Systems, 2017; 531(14): 105–121.
Virlet N, Sabermanesh K, Sadeghi-Tehran P, Hawkesford M J. Field Scanalyzer: An automated robotic field phenotyping platform for detailed crop monitoring. Functional Plant Biol, 2017; 44(1): 143. doi: 10.1071/ FP16163.
Weyler J, Milioto A, Falck T, Behley J, Stachniss C. Joint plant instance detection and leaf count estimation for in-field plant phenotyping. IEEE Robot Autom Lett, 2021; 6(2): 3599–3606.
Andrade-Sanchez P, Gore M A, Heun J T, Thorp K R, Carmo-Silva A E, French A N, et al. Development and evaluation of a field-based high-throughput phenotyping platform. Functional Plant Biol, 2014; 41(1): 68. doi: 10.1071/FP13126.
Sivakumar A, Modi S, Gasparino M, Ellis C, Baquero Velasquez A, Chowdhary G, et al. Learned visual navigation for under-canopy agricultural robots. Robotics: Science and Systems XVII, 2021. doi: 10.15607/RSS.2021.XVII.019.
Derby S J, McFadden J. Treadbot multi-head robot for high throughput applications. ASME International Mechanical Engineering Congress and Exposition, 2009; 43(772): 157–164.
Xiao D Y, Gong L, Liu C L, Huang Y X. Phenotypebased robotic screening platform for leafy plant breeding. IFACPapersOnLine, 2016; 49(16): 237–241.
Sabzi S, Abbaspour-Gilandeh Y, García-Mateos G, RuizCanales A, Molina-Martínez A M, Arribas J I. An automatic nondestructive method for the classification of the ripeness stage of red delicious apples in orchards using aerial video. Agronomy, 2019; 9(2): 84. doi: 10.3390/ agronomy9020084.
Bao Y, Tang L, Srinivasan S, Schnable P S. Field-based architectural traits characterisation of maize plant using time-of-flight 3D imaging. Biosystems Engineering, 2019; 178: 86–101.
Wang Y T, Li M Z, Ji R H, Wang M J, Zhang Y, Zheng L H. Novel encoder for ambient data compression applied to microcontrollers in agricultural robots. Int J Agric & Biol Eng, 2022; 15(4): 197–204.
Ma Z, Wang Y, Zhang T S, Wang H G, Jia Y J, Gao R, et al. Maize leaf disease identification using deep transfer convolutional neural networks. Int J Agric & Biol Eng, 2022; 15(5): 187–195.
Fan J C, Zhang Y, Wen W L, Gu S H, Lu X J, Guo X Y. The future of Internet of Things in agriculture: Plant high-throughput phenotypic platform. Journal of Cleaner Production, 2021; 280: 123651. doi: 10.1016/j.jclepro.2020.123651.
Magistri F, Marks E, Nagulavancha S, Vizzo I, Läebe T, Behley J, et al. Contrastive 3D shape completion and reconstruction for agricultural robots using RGB-D frames. IEEE Robotics and Automation Letters, 2022; 7: 10120–10127.
Kahraman S, Bacher R. A comprehensive review of hyperspectral data fusion with lidar and sar data. Annual Reviews in Control, 2021; 51: 236–253.
Shu M Y, Shen M Y, Zuo J Y, Yin P F, Wang M, Xie Z W, et al. The application of UAV-based hyperspectral imaging to estimate crop traits in maize inbred lines. Plant Phenomics, 2021; 2021: 9890745. doi: 10.34133/2021/9890745.
Zaman-Allah M, Vergara O, Araus J L, Tarekegne A, Magorokosho C, Zarco-Tejada P J, et al. Unmanned aerial platform-based multi-spectral imaging for field phenotyping of maize. Plant Methods, 2015; 11: 35. doi: 10.1186/s13007-015-0078-2.
Christiansen M, Laursen M, Jørgensen R, Skovsen S, Gislum R. Designing and Testing a UAV Mapping System for Agricultural Field Surveying. Sensors, 2017; 17(12): 2703. doi: 10.1186/s13007-015-0078-2.
Zermas D, Morellas D, Mulla D, Papanikolopoulos N. 3D model processing for high throughput phenotype extraction–the case of corn. Computers and Electronics in Agriculture, 2020; 172: 105047. doi:10.1016/j.compag.2019.105047.
Xiang L, Nolan T M, Bao Y, Elmore M, Tuel T, Gai J, et al. Robotic Assay for Drought (RoAD): An automated phenotyping system for brassinosteroid and drought responses. Plant J, 2021; 107(6): 1837–1853.
Yao L, van de Zedde R, Kowalchuk G. Recent developments and potential of robotics in plant eco-phenotyping. Jez JM, Topp CN, eds. Emerging Topics in Life Sciences, 2021; 5(2): 289–300.
Vit A, Shani G. Comparing RGB-D sensors for close range outdoor agricultural phenotyping. Sensors, 2018; 18(12): 4413. doi: 10.3390/ s18124413.
Buzzy M, Thesma V, Davoodi M, Mohammadpour Velni J. Real-time plant leaf counting using deep object detection networks. Sensors, 2020; 20(23): 6896. doi: 10.3390/s20236896.
Rincón M G, Mendez D, Colorado J D. Four-dimensional plant phenotyping model integrating low-density LiDAR data and multispectral images. Remote Sensing, 2022; 14(2): 356. doi: 10.3390/rs14020356.
Tian Y, Xie L, Wu M, Yang B, Ishimwe C, Ye D, et al. Multicolor fluorescence imaging for the early detection of salt stress in arabidopsis. Agronomy, 2021; 11(12): 2577. doi: 10.3390/agronomy11122577.
Zhang L, Maki H, Ma D, Sánchez-Gallego J A, Mickelbart M V, Wang L, et al. Optimized angles of the swing hyperspectral imaging system for single corn plant. Computers and Electronics in Agriculture, 2019; 156: 349–359.
Sampaio G S, Silva LA, Marengoni M. 3D reconstruction of non-rigid plants and sensor data fusion for agriculture phenotyping. Sensors, 2021; 21(12): 4115. doi: 10.3390/s21124115.
Zhang C, Gao H, Zhou J, Cousins A, Pumphrey M O, Sankaran S. 3D robotic system development for high-throughput crop phenotyping. IFAC-PapersOnLine, 2016; 49(16): 242–247.
Wang J S, Tao B, Gong Z Y, Yu W F, Yin Z P. A mobile robotic 3-D measurement method based on point clouds alignment for large-scale complex surfaces. IEEE Transactions on Instrumentation and Measurement, 2021; 70: 1–11.
Ninomiya S. High-throughput field crop phenotyping: current status and challenges. Breed Sci, 2022; 72(1): 3–18.
Luo B H, Yang J, Song S L, Shi S, Gong W, Wang A, et al. Target classification of similar spatial characteristics in complex urban areas by using multispectral LiDAR. Remote Sensing, 2022; 14(1): 238. doi: 10.3390/rs14010238.
Jiang Y, Li C Y, Paterson A H, Sun S P, Xu R, Robertson J. Quantitative analysis of cotton canopy size in field conditions using a consumer-grade RGB-D camera. Front Plant Sci, 2018; 8: 2233. doi: 1.3389/ fpls.2017.02233.
Gonzalez F, Castro M P G, Narayan P, Walker R, Zeller L. Development of an autonomous unmanned aerial system to collect time-stamped samples from the atmosphere and localize potential pathogen sources. J Field Robotics, 2011; 28(6): 961–976.
Bailey B N, Stoll R, Pardyjak E R, Miller N E. A new three-dimensional energy balance model for complex plant canopy geometries: Model development and improved validation strategies. Agricultural and Forest Meteorology, 2016; 218–219: 146-160.
Araus J L, Cairns J E. Field high-throughput phenotyping: The new crop breeding frontier. Trends in Plant Science, 2014; 19(1): 52–61.
Chapman S C, Merz T, Chan A, Jackway P, Hrabar S, Dreccer M F, et al. Phenocopter: A low-altitude, autonomous remote-sensing robotic helicopter for high-throughput field-based phenotyping. Agronomy, 2014; 4: 279–301.
Vázquez-Arellano M, Reiser D, Paraforos D, Garrido-Izard M, Griepentrog H. Leaf area estimation of reconstructed maize plants using a time-of-flight camera based on different scan directions. Robotics, 2018; 7(4): 63. doi: 10.3390/robotics7040063.
Guan H O, Liu M, Ma X D, Yu S. Three-dimensional reconstruction of soybean canopies using multisource imaging for phenotyping analysis. Remote Sensing, 2018; 10(8): 1206. doi: 10.3390/rs10081206.
Shi W N, van de Zedde R, Jiang H Y, Kootstra G. Plant-part segmentation using deep learning and multi-view vision. Biosystems Engineering, 2019; 187: 81–95.
Erdle K, Mistele B, Schmidhalter U. Comparison of active and passive spectral sensors in discriminating biomass parameters and nitrogen status in wheat cultivars. Field Crops Research, 2011; 124(1): 74–84.
Boogaard F P, Rongen K S A H, Kootstra G W. Robust node detection and tracking in fruit-vegetable crops using deep learning and multi-view imaging. Biosystems Engineering, 2020; 192: 117–132.
Colaço A F, Molin J P, Rosell-Polo J R, Escolà A. Application of light detection and ranging and ultrasonic sensors to high-throughput phenotyping and precision horticulture: current status and challenges. Hortic Res, 2018; 5(1): 480–490.
Duckett T, Pearson S, Blackmore S, Grieve B, Chen W H, Cielniak G, et al. Agricultural robotics: the future of robotic agriculture. ArXiv, 2018; 1806: 06762. doi: 10.48550/arXiv.1806.06762.
Wang B, Gao Y, Sun C, Blumenstein M, La Salle J. Can walking and measuring along chord bunches better describe leaf shapes? 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017; pp.2047–2056.
Oishi Y, Habaragamuwa H, Zhang Y, Sugiura R, Asano K, Akai K, et al. Automated abnormal potato plant detection system using deep learning models and portable video cameras. International Journal of Applied Earth Observation and Geoinformation, 2021; 104: 102509. doi: 10.1016/j.jag.2021.102509.
Kutnjak D, Tamisier L, Adams I, Boonham N, Candresse T, Chiumenti M, et al. A primer on the analysis of high-throughput sequencing data for detection of plant. Microorganisms, 2021; 9(4): 841. doi: 10.3390/microorganisms9040841.
Barnett J, Duke M, Au C K, Lim S H. Work distribution of multiple Cartesian robot arms for kiwifruit harvesting. Computers and Electronics in Agriculture, 2020; 169: 105202. doi: 10.1016/j.compag.2019.105202.
Huang J X, Gómez-Dans J L, Huang H, Ma H Y, Wu Q L, Lewis P E, et al. Assimilation of remote sensing into crop growth models: Current status and perspectives. Agricultural and Forest Meteorology, 2019; 276–277: 107609. doi: 10.1016/j.agrformet.2019.06.008.
Sun G X, Li Y B, Wang X C, Hu G Y, Wang X, Zhang Y. Image segmentation algorithm for greenhouse cucumber canopy under various natural lighting conditions. Int J Agric & Biol Eng, 2016; 9(3): 130–138.
Xiang L R, Bao Y, Tang L, Ortiz D, Salas-Fernandez M G. Automated morphological traits extraction for sorghum plants via 3D point cloud data analysis. Computers and Electronics in Agriculture, 2019; 162: 951–961.
Atefi A, Ge Y F, Pitla S, Schnable J. In vivo human-like robotic phenotyping of leaf traits in maize and sorghum in greenhouse. Computers and Electronics in Agriculture, 2019; 163: 104854. doi: 10.1016/j.compag.2019.104854.
He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016; pp.770–778. doi: 10.1109/CVPR.2016.90.
Dyrmann M, Christiansen P, Midtiby H S. Estimation of plant species by classifying plants and leaves in combination J Field Robotics, 2018; 35(2): 202–212.
Jing Z W, Guan H Y, Zhao P R, Li D L, Yu Y T, Zang Y F, et al. Multispectral LiDAR point cloud classification using SE-PointNet++. Remote Sensing, 2021; 13(13): 2516. doi: 10.3390/rs13132516.
Das S, Christopher J, Apan A, Roy Choudhury M, Chapman S, Menzies N W, et al. UAV-thermal imaging and agglomerative hierarchical clustering techniques to evaluate and rank physiological performance of wheat genotypes on sodic soil ISPRS Journal of Photogrammetry and Remote Sensing, 2021; 173: 221–237.
Elnashef B, Filin S, Lati R N. Tensor-based classification and segmentation of three-dimensional point clouds for organ-level plant phenotyping and growth analysis Computers and Electronics in Agriculture, 2019; 156: 51–61.
Cao X F, Yu K Q, Zhao Y R, Zhao H H. Current status of high-throughput plant phenotyping for abiotic stress by imaging spectroscopy: A review. Spectroscopy and Spectral Analysis, 2020; 40(11): 3365–3372. (in Chinese)
Jin Y C, Liu J Z, Xu Z J, Yuan S Q, Li P P, Wang J Z. Development status and trend of agricultural robot technology. Int J Agric & Biol Eng, 2021; 14(4): 1–19.
Peña J M, Torres-Sánchez J, de Castro A I, Kelly M, López-Granados F. Weed mapping in early-season maize fields using object-based analysis of unmanned aerial vehicle (UAV) images. Suarez O D, ed. PLoS One, 2013; 8(10): e77151. doi: 10.1371/journal.pone.0077151.
McCool C, Perez T, Upcroft B. Mixtures of Lightweight Deep Convolutional Neural Networks: Applied to agricultural robotics. IEEE Robot Autom Lett, 2017; 2(3): 1344–1351.
Aich S, Josuttes A, Ovsyannikov I, Strueby K, Ahmed I, Duddu H S, et al. DeepWheat: Estimating phenotypic traits from crop images with deep learning. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tohoe: IEEE, 2018; pp.323–332. doi: 10.1109/WACV.2018.00042.
Yin X, Li W H, Li Z, Yi L L. Recognition of grape leaf diseases using MobileNetV3 and deep transfer learning. Int J Agric & Biol Eng, 2022; 15(3): 184–194.
Ho Tong Minh D, Ienco D, Gaetano R, Lalande N, Ndikumana E, Osman F, et al. Deep recurrent neural networks for winter vegetation quality mapping via multitemporal SAR Sentinel-1. IEEE Geosci Remote Sensing Lett, 2018; 15(3): 464–468.
Li K, Wang S, Zhang X, Xu Y, Xu W, Tu Z. Pose recognition with cascade transformers. arXiv: 2021; 2104: 06976. doi: 2021. 10.48550/arXiv.2104.06976
Hairmansis A, Berger B, Tester M, Roy S J. Image-based phenotyping for non-destructive screening of different salinity tolerance traits in rice. Rice, 2014; 7(1): 16. doi: 10.1186/s12284-014-0016-3.
Li Y T, He L Y, Jia J M, Chen J N, Lyu J, Wu C Y. High-efficiency tea shoot detection method via a compressed deep learning model. Int J Agric & Biol Eng, 2022; 15(3): 159–166.
Simon M, Frédéric B, Benoit de S, Scott T, Dan D, Stéphane J, et al. High-throughput phenotyping of plant height: comparing unmanned aerial vehicles and ground LiDAR estimates. Frontiers in Plant Science, 2017; 8: 02002. doi: 10.3389/fpls.2017.02002.
Hosoi F, Nakabayashi K, Omasa K. 3-D modeling of tomato canopies using a high-resolution portable scanning Lidar for extracting structural information. Sensors, 2011; 11(2): 2166–2174.
Zhou S Z, Kang F, Li W B, Kan J M, Zheng Y J. Point cloud registration for agriculture and forestry crops based on calibration balls using Kinect V2. Int J Agric & Biol Eng, 2020; 13(1): 198–205.
Mueller-Sim T, Jenkins M, Abel J, Kantor G. The Robotanist: A ground-based agricultural robot for high-throughput crop phenotyping 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017; pp.3634–3639.
Ma L, Li M, Tong L, Wang Y, Cheng L. Using unmanned aerial vehicle for remote sensing application. In: 21st International Conference on Geoinformatics, 2013; pp.1–5.
Zhang C H, Kovacs J M. The application of small unmanned aerial systems for precision agriculture: A review. Precision Agric, 2012; 13(6): 693–712.
Prashar A, Jones H. Infra-red thermography as a high-throughput tool for field phenotyping. Agronomy, 2014; 4(3): 397-417.
Vidal V, Honório L, Pinto M, Dantas M, Aguiar M, Capretz M. An edge-fog architecture for distributed 3D reconstruction and remote monitoring of a power plant site in the context of 5G. Sensors, 2022; 22(12): 4494. doi: 10.3390/s22124494.
Shamshiri R R, Weltzien C, Hameed I A, Yule I J, Grift T E, Balasundram S K, et al. Research and development in agricultural robotics: A perspective of digital farming. Int J Agric & Biol Eng, 2018; 11(4): 1–14.
Lu W, Zeng M J, Qin H H. Intelligent navigation algorithm of plant phenotype detection robot based on dynamic credibility evaluation. Int J Agric & Biol Eng, 2021; 14(6): 195–206.
Copyright (c) 2023 International Journal of Agricultural and Biological Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.