Review of deep learning-based weed identification in crop fields

Wenze Hu, Samuel Oliver Wane, Junke Zhu, Dongsheng Li, Qing Zhang, Xiaoting Bie, Yubin Lan

Abstract


Automatic weed identification and detection are crucial for precision weeding operations. In recent years, deep learning (DL) has gained widespread attention for its potential in crop weed identification. This paper provides a review of the current research status and development trends of weed identification in crop fields based on DL. Through an analysis of relevant literature from both within and outside of China, the author summarizes the development history, research progress, and identification and detection methods of DL-based weed identification technology. Emphasis is placed on data sources and DL models applied to different technical tasks. Additionally, the paper discusses the challenges of time-consuming and laborious dataset preparation, poor generality, unbalanced data categories, and low accuracy of field identification in DL for weed identification. Corresponding solutions are proposed to provide a reference for future research directions in weed identification.
Keywords: deep learning, weed detection, weed classification, image segmentation, Convolutional Neural Network, image processing
DOI: 10.25165/j.ijabe.20231604.8364

Citation: Hu W Z, Wane S O, Zhu J K, Li D S, Zhang Q, Bie X T, et al. Review of deep learning-based weed identification in crop fields. Int J Agric & Biol Eng, 2023; 16(4): 1-10.

Keywords


deep learning, weed detection, weed classification, image segmentation, Convolutional Neural Network, image processing

Full Text:

PDF

References


Rehman T U, Mahmud M S, Chang Y K, Jin J, Shin J. Current and future applications of statistical machine learning algorithms for agricultural machine vision systems. Computers and Electronics in Agriculture, 2019; 156: 585-605.

Patel D, Kumbhar B. Weed and its management: A major threats to crop economy. Journal Pharmaceutical Science and Bioscientific Research (JPSBR), 2016; 6(6): 753-758.

Zimdahl R L. Fundamentals of weed science. Academic Press, 2018; 758p.

Iqbal N, Manalil S, Chauhan B S, Adkins S W. Investigation of alternate herbicides for effective weed management in glyphosate-tolerant cotton. Archives of Agronomy and Soil Science, 2019; 65(13): 1885-1899.

Oerke E-C. Crop losses to pests. The Journal of Agricultural Science, 2006; 144(1): 31-43.

Rodrigo M, Oturan N, Oturan M A. Electrochemically assisted remediation of pesticides in soils and water: A review. Chemical Reviews, 2014; 114(17): 8720-8745.

Nørremark M, Griepentrog H W, Nielsen J, Søgaard H T. The development and assessment of the accuracy of an autonomous GPS-based system for intra-row mechanical weed control in row crops. Biosystems Engineering, 2008; 101(4): 396-410.

Tillett N, Hague T, Grundy A, Dedousis A P. Mechanical within-row weed control for transplanted crops using computer vision. Biosystems Engineering, 2008; 99(2): 171-178.

Talukdar S, Singha P, Mahato S, Shahfahad, Pal S, Liou Y-A, et al. Land-use land-cover classification by machine learning classifiers for satellite observations - A review. Remote Sensing, 2020; 12(7): 1135. doi: 10.3390/rs12071135.

Hamuda E, Glavin M, Jones E. A survey of image processing techniques for plant extraction and segmentation in the field. Computers and Electronics in Agriculture, 2016; 125: 184-199.

Chaisattapagon N Z C. Effective criteria for weed identification in wheat fields using machine vision. Transactions of the ASAE, 1995; 38(3): 965-974.

Jafari A, Mohtasebi S S, Jahromi H E, Omid M. Weed detection in sugar beet fields using machine vision. International Journal of Agriculture and Biology, 2006; 8(5): 602-605.

Zheng Y, Zhu Q, Huang M, Guo Y, Qin J W. Maize and weed classification using color indices with support vector data description in outdoor fields. Computers and Electronics in Agriculture, 2017; 141: 215-222.

Kazmi W, Garcia-Ruiz F J, Nielsen J, Rasmussen J, Andersen H J. Detecting creeping thistle in sugar beet fields using vegetation indices. Computers and Electronics in Agriculture, 2015; 112: 10-19.

Bakhshipour A, Jafari A, Nassiri S M, Zare D. Weed segmentation using texture features extracted from wavelet sub-images. Biosystems Engineering, 2017; 157: 1-12.

Meyer G, Mehta T, Kocher M F, Mortensen D A, Samal A. Textural imaging and discriminant analysis for distinguishing weeds for spot spraying. Transactions of the ASAE, 1998; 41(4): 1189-1197.

Lecun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015; 521(7553): 436-444.

Patterson J, Gibson A. Deep learning: A practitioner's approach. " O'Reilly Media, Inc.", 2017; 576p.

Su W-H. Advanced machine learning in point spectroscopy, RGB-and hyperspectral-imaging for automatic discriminations of crops and weeds: A review. Smart Cities, 2020; 3(3): 767-792.

Kamilaris A, Prenafeta-Boldú F X. Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 2018; 147: 70-90.

Wang A, Zhang W, Wei X. A review on weed detection using ground-based machine vision and image processing techniques. Computers and Electronics in Agriculture, 2019; 158: 226-240.

Lv S, Li D, Xian R. Research status of deep learning of in agriculture China. Computer Engineering and Applications, 2019; 55(20): 24-33, 51. (in Chinese)

Weng Y, Zeng R, Wu C M, Wang M, Wang X J, Liu Y J. A survey on deep-learning-based plant phenotype research in agriculture. Scientia Sinica (Vitae), 2019; 49(6): 698-716. (in Chinese)

Sun H, Li S, Li M Z, Liu H J, Qiao L, Zhang Y. Research progress of image sensing and deep learning in agriculture. Transactions of the CSAM, 2020; 51(5): 1-17. (in Chinese)

Najafabadi M M, Villanustre F, Khoshgoftaar T M, Seliya N, Wald R, Muharemagic E. Deep learning applications and challenges in big data analytics. Journal of Big Data, 2015; 2(1): 1-21.

Goodfellow I, Bengio Y, Courville A. Deep learning. MIT Press, 2016; 767p.

Gardner M W, Dorling S. Artificial neural networks (the multilayer perceptron) - A review of applications in the atmospheric sciences. Atmospheric Environment, 1998; 32(14-15): 2627-2636.

Gu J X, Wang Z H, Kuen J, Ma L Y, Shahroudy A, Shuai B, et al. Recent advances in convolutional neural networks. Pattern Recognition, 2018, 77: 354-377.

Hinton G E, Osindero S, Teh Y-W. A fast learning algorithm for deep belief nets. Neural Computation, 2006; 18(7): 1527-1554.

Medsker L R, Jain L. Recurrent neural network. Design and Applications, 2001; 5: 64-67.

Aggarwal A, Mittal M, Battineni G. Generative adversarial network: An overview of theory and applications. International Journal of Information Management Data Insights, 2021; 1(1): 100004. doi: 10.1016/j.jjimei.2020.100004.

Zhang S, Tong H, Xu J, Maciejewski R. Graph convolutional networks: a comprehensive review. Computational Social Networks, 2019; 6(1): 1-23.

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Advances in neural information processing systems. Curran Associates, Inc, 2014; 27: 2672-2680.

Wu Z H, Pan S R, Chen F W, Long G D, Zhang C Q, et al. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020; 32(1): 4-24.

Yu J, Sharpe S M, Schumann A W, Yu P S. Deep learning for image-based weed detection in turfgrass. European Journal of Agronomy, 2019; 104: 78-84.

Lee W S, Alchanatis V, Yang C, Hirafuji M, Moshou D, Li C. Sensing technologies for precision specialty crop production. Computers and Electronics in Agriculture, 2010; 74(1): 2-33.

Ballesteros R, Ortega J F, Hernández D, Moreno M A, et al. Applications of georeferenced high-resolution images obtained with unmanned aerial vehicles. Part I: Description of image acquisition and processing. Precision Agriculture, 2014, 15(6): 579-592.

Zarco-Tejada P J, Diaz-Varela R, Angileri V, Loudjani P. Tree height quantification using very high resolution imagery acquired from an unmanned aerial vehicle (UAV) and automatic 3D photo-reconstruction methods. European Journal of Agronomy, 2014; 55: 89-99.

Huang H S, Lan Y B, Deng J Z, Yang A Q, Deng X L, Zhang L, et al. A semantic labeling approach for accurate weed mapping of high resolution UAV imagery. Sensors, 2018, 18(7): 2113. doi: 10.3390/s18072113.

Huang H S, Deng J Z, Lan Y B, Yang A Q, Deng X L, Zhang L. A fully convolutional network for weed mapping of unmanned aerial vehicle (UAV) imagery. PloS One, 2018; 13(4): e0196302. doi: 10.1471/journal.pone.0196302.

Huang H S, Lan Y B, Yang A Q, Zhang Y L, Wen S, Deng J Z. Deep learning versus Object-based Image Analysis (OBIA) in weed mapping of UAV imagery. International Journal of Remote Sensing, 2020; 41(9): 3446-3479.

Petrich L, Lohrmann G, Neumann M, Martin F, Frey A, Stoll A, et al. Detection of Colchicum autumnale in drone images, using a machine-learning approach. Precision Agriculture, 2020; 21(6): 1291-1303.

Osorio K, Puerto A, Pedraza C, Jamaica D, Rodriguez L. A deep learning approach for weed detection in lettuce crops using multispectral images. AgriEngineering, 2020; 2(3): 471-488.

Adhikari S P, Yang H, Kim H. Learning semantic graphics using convolutional encoder–decoder network for autonomous weeding in paddy. Frontiers in Plant Science, 2019; 10: 1404. doi: 10.3389/fpls.2019/01403.

Gao J, French A P, Pound M P, He Y, Pridmore T P, Pieters J G. Deep convolutional neural networks for image-based Convolvulus sepium detection in sugar beet fields. Plant Methods, 2020; 16(1): 1-12.

Ma X, Deng X W, Qi L, Jiang Y, Li H W, Wang Y W, et al. Fully convolutional network for rice seedling and weed image segmentation at the seedling stage in paddy fields. PloS One, 2019, 14(4): e0215676. doi: 10.1371/journal.pone.0215676.

Teimouri N, Dyrmann M, Nielsen P R, Mathiassen S K, Somerville G J, Jorgensen R N, et al. Weed growth stage estimator using deep convolutional neural networks. Sensors, 2018; 18(5): 1580. doi: 10.3390/s18051580.

Yu J, Schumann A W, Cao Z, Sharpe S M, Boyd N S. Weed detection in perennial ryegrass with deep learning convolutional neural network. Frontiers in Plant Science, 2019; 10: 1422. doi: 10.3389/fpls.2019.01422.

Farooq A, Hu J, Jia X. Analysis of spectral bands and spatial resolutions for weed classification via deep convolutional neural network. IEEE Geoscience and Remote Sensing Letters, 2018; 16(2): 183-187.

Farooq A, Jia X, Hu J, Zhou J. Multi-resolution weed classification via convolutional neural network and superpixel based local binary pattern using remote sensing images. Remote Sensing, 2019; 11(14): 1692. doi: 10.3390/rs11141692.

Farooq A, Hu J, Jia X. Weed classification in hyperspectral remote sensing images via deep convolutional neural network. In: IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, 2018; pp.3816-3819.

Chebrolu N, Lottes P, Schaefer A, Winterhalter W, Burgard W, Stachniss C. Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields. International Journal of Robotics Research, 2017; 36(10): 1045-1052.

Sudars K, Jasko J, Namatevs I, Ozola L, Badukis N. Dataset of annotated food crops and weed images for robotic computer vision control. Data in Brief, 2020; 31: 105833. doi: 10.1016/j.dib.2020.105833.

Leminen Madsen S, Mathiassen S K, Dyrmann M, Lauren M S, Paz L-C, Jorgensen R N. Open plant phenotype database of common weeds in Denmark. Remote Sensing, 2020; 12(8): 1246. doi: 10.3390/rs12081246.

Wang A C, Zhang W, Wei X H. A review on weed detection using ground-based machine vision and image processing techniques. Computers and Electronics in Agriculture, 2019; 158: 226-240.

Zhang X-Y, Shi H C, Zhu X B, Li P. Active semi-supervised learning based on self-expressive correlation with generative adversarial networks. Neurocomputing, 2019; 345: 103-113.

Berge T, Aastveit A, Fykse H. Evaluation of an algorithm for automatic detection of broad-leaved weeds in spring cereals. Precision Agriculture, 2008; 9(6): 391-405.

Liu B, Bruch R. Weed detection for selective spraying: a review. Current Robotics Reports, 2020; 1(1): 19-26.

Zhang L, Jin X, Fu L Y, Li S W. Recognition method for weeds in rapeseed field based on Faster R-CNN deep network. Laser & Optoelectronics Progress, 2020; 57(2): 304-312. (in Chinese)

Xu Y, Wen D S, Zhou J P, Fan X P, Liu Y. Identification method of cotton seedlings and weeds in Xinjiang based on Faster R-CNN. Journal of Drainage and Irrigation Machinery Engineering, 2021; 39(6): 602-607. (in Chinese)

Peng M X, Xia J F, Peng H. Efficient recognition of cotton and weed in field based on Faster R-CNN by integrating FPN. Transactions of the CSAE, 2019; 35(20): 202-209. (in Chinese)

Deng X W, Qi L, Ma X, Jiang Y, Chen X S, Liu H Y, et al. Recognition of weeds at seedling stage in paddy fields using multi-feature fusion and deep belief networks. Transactions of the CSAE, 2018; 34(14): 165-172. (in Chinese)

Peng W, Lan Y B, Yue X J, Cheng Z Y, Wang L H, Cen Z L, et al. Research on paddy weed recognition based on deep convolutional neural network. Journal of South China Agricultural University, 2020; 41(6): 75-81. (in Chinese)

Espejo-Garcia B, Mylonas N, Athanasakos L, Fountas S, Vasilakoglou I. Towards weeds identification assistance through transfer learning. Computers and Electronics in Agriculture, 2020; 171: 105306. doi: 10.1016/j.compag.2020.105306.

Espejo-Garcia B, Mylonas N, Athanasakos L, Vali E, Fountas S. Combining generative adversarial networks and agricultural transfer learning for weeds identification. Biosystems Engineering; 2021; 204: 79-89.

Chen D, Lu Y Z, Li Z J, Young S. Performance evaluation of eeep transfer learning on multiclass identification of common weed species in cotton production systems. arXiv preprint; 2021. arXiv: 2110.04960.

Dyrmann M, Karstoft H, Midtiby H S. Plant species classification using deep convolutional neural network. Biosystems engineering, 2016; 151: 72-80.

Xu Y, He R, Zhai Y, Zhao B, Li C. Weed identification method based on deep transfer learning in field natural environments. Journal of Jilin University (Engineering and Technology Edition), 2021; 51(6): 2304-2312.

Kumar J D, Babu C G, Priyadharsini K. An experimental investigation to spotting the weeds in rice field using deepnet. Materials Today: Proceedings, 2021; 45: 8041-8053.

Partel V, Kakarla S C, Ampatzidis Y. Development and evaluation of a low-cost and smart technology for precision weed management utilizing artificial intelligence. Computers and electronics in agriculture, 2019; 157: 339-350.

Ahmad A, Saraswat D, Aggarwal V, Etienne A, Hancock B. Performance of deep learning models for classifying and detecting common weeds in corn and soybean production systems. Computers and Electronics in Agriculture, 2021; 184: 106081. doi: 10.1016/j.compag.2021.106081.

Zhang R F, Wang C, Hu X P, Liu Y X, Chen S, Su B F. Weed location and recognition based on UAV imaging and deep learning. International Journal of Precision Agricultural Aviation, 2020; 3(1): 23-29.

Yu G, Jiang H, Sun T, Wang C, Shang J. Weed Identification in Cabbage Field Based on Deep Learning. Software, 2020; 41(4): 211-215. (in Chinese)

Champ J, Mora‐Fallas A, Goëau H, Mata-Montero E, Bonnet P, Joly A. Instance segmentation for the fine detection of crop and weed plants by precision agricultural robots. Applications in Plant Sciences, 2020; 8(7): e11373. doi: 10.1002/aps3.11373.

Quan L, Wu B, Mao S. A Mask R-CNN-based method for weed instance segmentation and leaf age recognition in agricultural fields. Journal of Northeast Agricultural University, 2021; 52(4): 65-76. (in Chinese)

Abdalla A, Cen H Y, Wan L, Rashid R, Weng H Y, Zhou W J, et al. Fine-tuning convolutional neural network with transfer learning for semantic segmentation of ground-level oilseed rape images in a field with high weed pressure. Computers and Electronics in Agriculture, 2019; 167: 105091. doi: 10.1016/j.compag.2019/105091.

Asad M H, Bais A. Weed detection in canola fields using maximum likelihood classification and deep convolutional neural network. Information Processing in Agriculture, 2020; 7(4): 535-545.

Shang J, Jiang H, Yu G, Chen Z, Wang B, Li Z, et al. Weed identification system based on deep learning. Software Guide, 2020; 19(7): 127-130. (in Chinese)




Copyright (c) 2023 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office