Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles

Xin Zhao, Wanli Wang, Long Wen, Zhibo Chen, Sixian Wu, Kun Zhou, Mengyao Sun, Lanjun Xu, Bingbing Hu, Caicong Wu

Abstract


Digital twins can improve the level of control over physical entities and help manage complex systems by integrating a range of technologies. The autonomous agricultural machine has shown revolutionary effects on labor reduction and utilization rate in field works. Autonomous vehicles in precision agriculture have the potential to improve competitiveness compared to current crop production methods and have become a research hotspot. However, the development time and resources required in experiments have limited the research in this area. Simulation tools in unmanned farming that are required to enable more efficient, reliable, and safe autonomy are increasingly demanding. Inspired by the recent development of an open-source virtual simulation platform, this study proposed an autoware-based simulator to evaluate the performance of agricultural machine guidance based on digital twins. Oblique photogrammetry using drones is used to construct three-dimensional maps of fields at the same scale as reality. A communication format suitable for agricultural machines was developed for data input and output, along with an inter-node communication methodology. The conversion, publishing, and maintenance of multiple coordinate systems were completed based on ROS (Robot Operating System). Coverage path planning was performed using hybrid curves based on Bézier curves, and it was tested in both a simulation environment and actual fields with the aid of Pure Pursuit algorithms and PID controllers.
Keywords: autoware, simulation platform, autonomous agricultural vehicle, digital twin; autonomous robots
DOI: 10.25165/j.ijabe.20231604.8039

Citation: Zhao X, Wang W L, Wen L, Chen Z B, Wu S X, Zhou K, et al. Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles. Int J Agric & Biol Eng, 2023; 16(4): 185-190.

Keywords


autoware, simulation platform, autonomous agricultural vehicle, digital twin; autonomous robots

Full Text:

PDF

References


Bochtis D D, Sørensen C G C, Busato P. Advances in agricultural machinery management: A review. Biosystems Engineering, 2014; 126: 69–81.

Moysiadis V, Tsolakis N, Katikaridis D, Sørensen C G, Pearson S, Bochtis D. Mobile robotics in agricultural operations: A narrative review on planning aspects. Applied Sciences, 2020; 10(10): 3453. doi: 10.3390/app10103453.

Oksanen T, Visala A. Coverage path planning algorithms for agricultural field machines. Journal of Field Robotics, 2009; 26(8): 651–668.

Bochtis D D, Vougioukas S G. Minimising the non-working distance travelled by machines operating in a headland field pattern. Biosystems Engineering, 2008; 101(1): 1–12.

Jensen K, Larsen M, Nielsen S H, Larsen L B, Olsen K S, Jørgensen R N. Towards an open software platform for field robots in precision agriculture. Robotics, 2014; 3(2): 207–234.

Zhang Y H, Wang J, Wang X N, Dolan J M. Road-segmentation-based curb detection method for self-driving via a 3D-LiDAR sensor. IEEE Trans Intell Transp Syst, 2018; 19(12): 3981–3991.

Yao L J, Hu D, Yang Z D, Li H B, Qian M B. Depth recovery for unstructured farmland road image using an improved SIFT algorithm. Int J Agric Biol Eng, 2019; 12(4): 141–147.

Yang L L, Li Y B, Chang M S, Xu Y Y, Hu B B, Wang X X, et al. Recognition of field roads based on improved U-Net plus plus Network. Int J Agric & Biol Eng, 2023; 16(2): 171–178.

Grieves M. Digital twin : Manufacturing excellence through virtual factory replication. Digital Twin White Papaper, 2014; 7p.

Kritzinger W, Karner M, Traar G, Henjes J, Sihn W. Digital Twin in manufacturing: A categorical literature review and classification. IFAC-PapersOnLine, 2018; 51(11): 1016–1022.

Caputo F, Greco A, Fera M, Macchiaroli R. Digital twins to enhance the integration of ergonomics in the workplace design. International Journal of Industrial Ergonomics, 2019; 71: 20–31.

Sivalingam K, Sepulveda M, Spring M, Davies P. A review and methodology development for remaining useful life prediction of offshore fixed and floating wind turbine power converter with digital twin technology perspective. In: 2018 2nd International Conference on Green Energy and Applications, 2018; pp.197–204.

Rong G D, Shin B H, Tabatabaee H, Lu Q, Lemke S, Možeiko M, et al. LGSVL simulator: A high fidelity simulator for autonomous driving. 2020; pp.1-6. doi: 10.1109/ITSC45102.2020.9294422.

Pylianidis C, Osinga S, Athanasiadis I N. Introducing digital twins to agriculture. Computers and Electronics in Agriculture, 2021; 184: 105942. doi: 10.1016/j.compag.2020.105942.

Zhao Y P, Lian L K, Bi C W, Xu Z J. Digital twin for rapid damage detection of a fixed net panel in the sea. Computers and Electronics in Agriculture, 2022; 200: 107247. doi: 10.1016/j.compag.2022.107247.

Tsolakis N, Bechtsis D, Bochtis D. Agros: A robot operating system based emulation tool for agricultural robotics. Agronomy, 2019; 9(7): 403. doi: 10.3390/agronomy9070403.

Dosovitskiy A, Ros G, Codevilla F, López A, Koltun V. CARLA: An open urban driving simulator. In: 1st Conference on Robot Learning (CoRL 2017), PMLR; 2017; 78: 1–16.

Ziegler S, Höpler R. Extending the IPG CarMaker by FMI compliant units. In: Proceedings of the 8th International Modelica Conference, Dresden: Technical Univeristy, Citeseer, 2011; pp.779–783. doi: 10.3384/ecp11063779.

Benekohal R F, Treiterer J. CARSIM: Car-following model for simulation of traffic in normal and stop-and-go conditions. Transportation Research Record Journal of the Transportation Research Board, 1988; 1194: 99–111.

Ryan R R. ADAMS—Multibody system analysis software. In: Multibody Systems Handbook, Springer, 1990; pp.361–402.

Yang Y, Li Y K, Wen X, Zhang G, Ma Q L, Cheng S K, et al. An optimal goal point determination algorithm for automatic navigation of agricultural machinery: Improving the tracking accuracy of the Pure Pursuit algorithm. Computers and Electronics in Agriculture, 2022; 194: 106760. doi: 10.1016/j.compag.2022.106760.

Foote T. tf: The transform library. In: 2013 IEEE Conference Technologies Practical Robot Applications (TePRA), Woburn: IEEE, 2013; pp.1–6. doi: 10.1109/TePRA.2013.6556373.

Hu H M, Kaizu Y, Zhang H D, Xu Y W, Imou K, Li M, et al. Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN. Int J Agric & Biol Eng, 2022; 15(6): 175–179.

Bao X L, Leng J S, Mao J C, Chen B Y. Cutting of sheep carcass using 3D point cloud with dual-robot system. Int J Agric Biol Eng, 2022; 15(5): 163–171.

Omasa K, Ono E, Ishigami Y, Shimizu Y, Araki Y. Plant functional remote sensing and smart farming applications. Int J Agric & Biol Eng, 2022; 15(4): 1–6.




Copyright (c) 2023 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office