Recognition of field roads based on improved U-Net++ Network

Lili Yang, Yuanbo Li, Mengshuai Chang, Yuanyuan Xu, Bingbing Hu, Xinxin Wang, Caicong Wu

Abstract


Unmanned driving of agricultural machinery has garnered significant attention in recent years, especially with the development of precision farming and sensor technologies. To achieve high performance and low cost, perception tasks are of great importance. In this study, a low-cost and high-safety method was proposed for field road recognition in unmanned agricultural machinery. The approach of this study utilized point clouds, with low-resolution lidar point clouds as inputs, generating high-resolution point clouds and Bird's Eye View (BEV) images that were encoded with several basic statistics. Using a BEV representation, road detection was reduced to a single-scale problem that could be addressed with an improved U-Net++ neural network. Three enhancements were proposed for U-Net++: 1) replacing the convolutional kernel in the original U-Net++ with an Asymmetric Convolution Block (ACBlock); 2) adding a multi-branch Asymmetric Dilated Convolutional Block (MADC) in the highest semantic information layer; 3) adding an Attention Gate (AG) model to the long-skip-connection in the decoding stage. The results of experiments of this study showed that our algorithm achieved a Mean Intersection Over Union of 96.54% on the 16-channel point clouds, which was 7.35 percentage points higher than U-Net++. Furthermore, the average processing time of the model was about 70 ms, meeting the time requirements of unmanned driving in agricultural machinery. The proposed method of this study can be applied to enhance the perception ability of unmanned agricultural machinery thereby increasing the safety of field road driving.
Keywords: image segmentation, unmanned agricultural machinery, field roads, point cloud super-resolution, point cloud bird's eye view
DOI: 10.25165/j.ijabe.20231602.7941

Citation: Yang L L, Li Y B, Chang M S, Xu Y Y, Hu B B, Wang X X, et al. Recognition of field roads based on improved U-Net++ Network. Int J Agric & Biol Eng, 2023; 16(2): 171-178.

Keywords


image segmentation, unmanned agricultural machinery, field roads, point cloud super-resolution, point cloud bird's eye view

Full Text:

PDF

References


Cui X Z, Feng Q, Wang S Z, Zhang J H. Monocular depth estimation with self-supervised learning for vineyard unmanned agricultural vehicle. Sensors, 2022; 22(3): 721. doi: 10.3390/s22030721.

Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.

He Y, Jiang H, Fang H, Wang Y, Liu Y F. Research progress of intelligent obstacle detection methods of vehicles and their application on agriculture. Transactions of the CSAE, 2018; 34(9): 21-32. (in Chinese)

Yao L J, Hu D, Yang Z D, Li H B, Qian M B. Depth recovery for unstructured farmland road image using an improved SIFT algorithm. Int J Agric & Biol Eng, 2019; 12(4): 141-147.

Pang S, Morris D, Radha H. CLOCs: Camera-LiDAR object candidates fusion for 3D object detection. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Las Vegas: IEEE, 2020; pp.10386-10393. doi: 10.1109/IROS45743.2020.9341791.

Cui Y P, Xu H, Wu J Q, Sun Y, Zhao J X. Automatic vehicle tracking with roadside LiDAR data for the connected-vehicles system. IEEE Intelligent Systems, 2019; 34(3): 44-51.

Lyu Y C, Bai L, Huang X M. Real-time road segmentation using lidar data processing on an FPGA. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2018; pp.1-5.

Kisner H, Thomas U. Segmentation of 3D point clouds using a new spectral clustering algorithm without a-priori knowledge. In: VISIGRAPP 2018, 2018; pp.315-322.

Zhang W. Lidar-based road and road-edge detection. In: 2010 IEEE Intelligent Vehicles Symposium. La Jolla: IEEE, 2010; pp.845-848. doi: 10.1109/IVS.2010.5548134.

Charles R Q, Su H, Mo K C, Guibas L J. Pointnet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017; pp.652-660.

Beltrán J, Guindel C, Moreno F M, Cruzado D, Garcia F, De La Escalera A. Birdnet: A 3D object detection framework from lidar information. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), IEEE, 2018; pp.3517-3523.

Hua B S, Tran M K, Yeung S K. Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018; pp.984-993.

Zhang Y H, Wang J, Wang X N, Dolan J M. Road-segmentation-based curb detection method for self-driving via a 3D-LiDAR sensor. IEEE Transactions on Intelligent Transportation Systems, 2018; 19(12): 3981-3991.

Zhou Y, Tuzel O. Voxelnet: End-to-end learning for point cloud based 3D object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, IEEE, 2018; pp.4490-4499.

Chang Y C, Xue F, Sheng F, Liang W T, Ming A L. Fast road segmentation via uncertainty-aware symmetric network. In: 2022 International Conference on Robotics and Automation (ICRA), Philadelphia: IEEE, 2022; pp.1124-11130. doi: 1109/ICRA46639.2022.9812452.

Zhu Z, Liu J L. Graph-based ground segmentation of 3D LIDAR in rough area. In: 2014 IEEE International Conference on Technologies for Practical Robot Applications (TePRA), IEEE, 2014; pp.1–5. doi: 10.1109/TePRA.2014.6869157.

Cheng Z Y, Ren G Q, Zhang Y. Ground segmentation from 3D point cloud using features of scanning line segments. Opto-Electronic Engineering, 2019; 46(7): 180268. doi: 10.12086/OEE.2019.180268.

Triess L T, Peter D, Rist C B, Enzweiler M, Zollner J M. CNN-based synthesis of realistic high-resolution LiDAR data. In: 2019 IEEE Intelligent Vehicles Symposium (IV), Paris: IEEE, 2019; pp.1512-1519.

Shan T X, Wang J K, Chen F F, Szenher P, Englot B Simulation-based lidar super-resolution for ground vehicles. Robotics and Autonomous Systems, 2020 134: 103647. doi: 10.1016/J.ROBOT.2020.103647.

Zhou Z, Siddiquee M M R, Tajbakhsh N, Liang J M. Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging, 2019; 39(6): 1856-1867.

Ding G G, Han J G, Ding X H, Guo Y C. ACNet: Strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, IEEE, 2019; pp.1911-1920.

Oktay O, Schlemper J, Folgoc L L, Lee M, Heinrich M, Misawa K, et al. Attention U-Net: Learning where to look for the pancreas. arXiv preprint, 2018; arXiv:1804.03999, 2018.

Yang J D, Zhu J T, Wang H L, Yang X. Dilated MultiResUNet: Dilated multiresidual blocks network based on U-Net for biomedical image segmentation. Biomedical Signal Processing and Control, 2021; 68: 102643. doi: 10.1016/j.bspc.2021.102643.

Bala S A, Kant S. Dense dilated inception network for medical image segmentation. International Journal of Advanced Computer Science and Applications (IJACSA), 2020; 11(11): 0111195. doi: 10.14569/IJACSA.2020.0111195.




Copyright (c) 2023 International Journal of Agricultural and Biological Engineering

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

2023-2026 Copyright IJABE Editing and Publishing Office