智能化农业装备学报（中英文）

基于不同拍摄角度多幅图像的果园视觉导航

预览PDF | 下载PDF

马锃宏 ¹^,²^,³ , 岳佳文 ¹^,²^,³ , 殷程 ¹^,²^,³ , 赵润茂 ¹^,²^,³ , Mulongoti CHANDA ¹^,²^,³ , 杜小强 ¹^,²^,³^,^*

智能化农业装备学报（中英文） | 2024,5(4): 51-65

收起

智能化农业装备学报（中英文） 2024 , 5 (4) : 51 -65

基于不同拍摄角度多幅图像的果园视觉导航

全屏

马锃宏¹^,²^,³, 岳佳文¹^,²^,³, 殷程¹^,²^,³, 赵润茂¹^,²^,³, Mulongoti CHANDA¹^,²^,³, 杜小强¹^,²^,³^,^*

作者信息

^1.浙江理工大学机械工程学院，浙江杭州，310018

^2.浙江省种植装备技术重点实验室，浙江杭州，310018

^3.农业农村部东南丘陵山区农业装备重点实验室（部省共建），浙江杭州，310018

通讯作者:

杜小强

Visual navigation in orchard based on multiple images at different shooting angles

Zenghong MA¹^,²^,³, Jiawen YUE¹^,²^,³, Cheng YIN¹^,²^,³, Runmao ZHAO¹^,²^,³, Mulongoti CHANDA¹^,²^,³, Xiaoqiang DU¹^,²^,³^,^*

Affiliations

^1.School of Mechanical Engineering, Zhejiang Sci-Tech University, Hangzhou 310018, China

^2.Key Laboratory of Transplanting Equipment and Technology of Zhejiang Province, Hangzhou 310018, China

^3.Key Laboratory of Agricultural Equipment for Hilly and Mountainous Areas in Southeastern China (Co-construction by Ministry and Province), Ministry of Agriculture and Rural Affairs, Hangzhou 310018, China

Corresponding Author:

^* DU Xiaoqiang, PhD, Professor, research interests: machine vision and agricultural robot, agricultural machinery equipment design. E-mail: xqiangdu@zstu.edu.cn

MA Zenghong, PhD, Associate Professor, research interests: agricultural machinery navigation and unmanned driving. E-mail: mzh2018@zstu.edu.cn

出版时间: 2024-11-15 doi: 10.12398/j.issn.2096-7217.2024.04.004

文章导航

摘要

收起

果园通常地形崎岖，树冠茂密，杂草丛生。由于信号遮挡、多径效应和射频干扰，GNSS难以在果园中进行自主导航。为了实现果园的自主导航，本文提出了一种基于不同拍摄角度下多幅图像的视觉导航方法。一种动态图像捕捉设备是为安装相机而设计的，可以在不同角度拍摄多个图像。首先，将获得的果园图像分为天空和土壤检测阶段。每个图像被转换到HSV空间，并通过中值滤波和形态学处理初步分割为天空、树冠和土壤区域。其次，利用最大连通区域算法提取天空和土壤区域，并利用Canny算子对区域边缘进行检测和滤波。第三，通过对区域坐标点的拟合，提取当前帧数中的导航线。然后使用动态加权滤波算法分别提取土壤和天空检测阶段的导航线，并将天空检测阶段导航线镜像到土壤区域。最后，利用卡尔曼滤波算法对最终导航路径进行融合提取。对200幅图像的测试结果表明，视觉导航路径拟合的准确率为95.5%，单帧图像处理耗时60 ms，满足导航的实时性和鲁棒性要求。在油茶园进行的视觉导航实验表明，当行驶速度为0.6 m/s时，无杂草和杂草环境下视觉导航的最大跟踪偏移量分别为0.14 m和0.24 m，均方根误差分别为30 mm和55 mm。

关键词

果园 / 视觉导航 / 多拍摄角度 / 区域分割 / 卡尔曼滤波器

Abstract

收起

The orchards usually have rough terrain, dense tree canopy and weeds. It is hard to use GNSS for autonomous navigation in orchard due to signal occlusion, multipath effect, and radio frequency interference. To achieve autonomous navigation in orchard, a visual navigation method based on multiple images at different shooting angles is proposed in this paper. A dynamic image capturing device is designed for camera installation and multiple images can be shot at different angles. Firstly, the obtained orchard images are classified into sky and soil detection stage. Each image is transformed to HSV space and initially segmented into sky, canopy and soil regions by median filtering and morphological processing. Secondly, the sky and soil regions are extracted by the maximum connected region algorithm, and the region edges are detected and filtered by the Canny operator. Thirdly, the navigation line in the current frame is extracted by fitting the region coordinate points. Then the dynamic weighted filtering algorithm is used to extract the navigation line for the soil and sky detection stage, respectively, and the navigation line for the sky detection stage is mirrored to the soil region. Finally, the Kalman filter algorithm is used to fuse and extract the final navigation path. The test results on 200 images show that the accuracy of visual navigation path fitting is 95.5%, and single frame image processing costs 60 ms, which meets the real-time and robustness requirements of navigation. The visual navigation experiments in Camellia oleifera orchard show that when the driving speed is 0.6 m/s, the maximum tracking offset of visual navigation in weed-free and weedy environments is 0.14 m and 0.24 m, respectively, and the RMSE is 30 mm and 55 mm, respectively.

Key words

orchard / visual navigation / multiple shooting angles / region segmentation / Kalman filter

引用本文

马锃宏, 岳佳文, 殷程, 赵润茂, Mulongoti CHANDA, 杜小强. 基于不同拍摄角度多幅图像的果园视觉导航. 智能化农业装备学报（中英文）, 2024 , 5 (4) : 51 -65 . DOI: 10.12398/j.issn.2096-7217.2024.04.004

Zenghong MA, Jiawen YUE, Cheng YIN, Runmao ZHAO, Mulongoti CHANDA, Xiaoqiang DU. Visual navigation in orchard based on multiple images at different shooting angles[J]. Journal of Intelligent Agricultural Mechanization, 2024 , 5 (4) : 51 -65 . DOI: 10.12398/j.issn.2096-7217.2024.04.004

基金

收起

参考文献

收起

文献

收起

参考文献引证文献

排序方式：

[1]

VERBIEST R, RUYSEN K, VANWALLEGHEM T, et al. Automation and robotics in the cultivation of pome fruit: Where do we stand today? [J]. Journal of Field Robotics, 2020.

[2]

MARINOUDI V, CLAUS G. SØRENSEN, PEARSON S, et al. Robotics and labour in agriculture. A context consideration [J]. Biosystems Engineering, 2019, 184: 111-121.

[3]

FUMIOMI T, WEI Y, CHANGYING L, et al. Applying new technologies to transform blueberry harvesting [J]. Agronomy, 2017, 7(2): 33.

[4]

NAVAS E, ROEMI FERNÁNDEZ, DELIA SEPÚLVEDA, et al. Soft grippers for automatic crop harvesting: A review [J]. Sensors, 2021, 21(8): 2689.

[5]

YANG Q, DU X, WANG Z, et al. A review of core agricultural robot technologies for crop productions [J]. Computers and Electronics in Agriculture, 2023, 206: 107701.

[6]

LECHNER W, BAUMANN S. Global navigation satellite systems [J]. Computers and Electronics in Agriculture, 2000, 25: 67-85.

[7]

GAN-MOR S, CLARK R L, UPCHURCH B L. Implement lateral position accuracy under RTK-GPS tractor guidance [J]. Computers and Electronics in Agriculture, 2007, 59: 31-38.

[8]

LIAO J, WANG Y, YIN J, et al. An integrated navigation method based on an adaptive federal Kalman filter for a rice transplanter [J]. Transactions of the ASABE, 2021, 64(2): 389-399.

[9]

HELLSTRÖM T. Autonomous navigation for forest machines [J]. Pre-Study Report, Department of Computing Science, Umea University, Sweden, 2002.

[10]

LI M, IMOU K, WAKABAYASHI K, et al. Review of research on agricultural vehicle autonomous guidance [J]. International Journal of Agricultural and Biological Engineering, 2009, 2(3): 1-16.

[11]

BENGOCHEA-GUEVARA J M, CONESA-MUÑOZ J, ANDÚJAR D, et al. Merge fuzzy visual servoing and GPS-based planning to obtain a proper navigation behavior for a small crop-inspection robot [J]. Sensors, 2016, 16(3): 276.

[12]

OPIYO S, OKINDA C, ZHOU J, et al. Medial axis-based machine-vision system for orchard robot navigation [J]. Computers and Electronics in Agriculture, 2021, 185: 106153.

[13]

SHARIFI M, CHEN X Q. A novel vision based row guidance approach for navigation of agricultural mobile robots in orchards [C]// 2015 6th International Conference on Automation, Robotics and Applications (ICARA). IEEE, 2015: 251-255.

[14]

RADCLIFFE J, COX J, BULANON D M. Machine vision for orchard navigation [J]. Computers in Industry, 2018, 98: 165-171.

[15]

ZHOU M, XIA J, YANG F, et al. Design and experiment of visual navigated UGV for orchard based on Hough matrix and RANSAC [J]. International Journal of Agricultural and Biological Engineering, 2021, 14(6): 176-184.

[16]

MALAVAZI F B P, GUYONNEAU R, FASQUEL J B, et al. LiDAR-only based navigation algorithm for an autonomous agricultural robot [J]. Computers and Electronics in Agriculture, 2018, 154: 71-79.

[17]

BLOK P M, SUH H K, VAN BOHEEMEN K, et al. Autonomous in-row navigation of an orchard robot with a 2D LIDAR scanner and particle filter with a laser-beam model [J]. Journal of Institute of Control, Robotics and Systems, 2018, 24(8): 726-735.

[18]

HAN Z H, LI J, YUAN Y W, et al. Path recognition of orchard visual navigation based on U-Net [J]. Transactions of the Chinese Society of Agricultural Machinery, 2021, 52(1): 30-39.

[19]

BI S, WANG Y H. Inter-line pose estimation and fruit tree location method for orchard robot [J]. Transactions of the Chinese Society of Agricultural Machinery, 2021, 52(8): 1-26, 39.

[20]

YANG Y, ZHANG Y L, MIAO W, et al. Accurate identification and location of corn rhizome based on faster R-CNN [J]. Transactions of the Chinese Society of Agricultural Machinery, 2018, 49(10): 46-53.

[21]

BLOK P M, VAN BOHEEMEN K, VAN EVERT F K, et al. Robot navigation in orchards with localization based on Particle filter and Kalman filter [J]. Computers and Electronics in Agriculture, 2019, 157: 261-269.

[22]

BERGERMAN M, MAETA S M, ZHANG J, et al. Robot farmers: Autonomous orchard vehicles help tree fruit production [J]. IEEE Robotics & Automation Magazine, 2015, 22(1): 54-63.

[23]

SHALAL N, LOW T, MCCARTHY C, et al. Orchard mapping and mobile robot localisation using on-board camera and laser scanner data fusion-Part B: Mapping and localisation [J]. Computers and Electronics in Agriculture, 2015, 119: 267-278.

[24]

MA Z, YIN C, DU X, et al. Rice row tracking control of crawler tractor based on the satellite and visual integrated navigation [J]. Computers and Electronics in Agriculture, 2022, 197: 106935.

[25]

HAN Leng, HE Xiongkui, WANG Changling, et al. Key technologies and equipment for smart orchard construction and prospects [J]. Smart Agriculture, 2022, 4(3): 1-11.

[26]

LIU Limin, HE Xiongkui, LIU Weihong, et al. Autonomous navigation and automatic target spraying robot for orchards [J]. Smart Agriculture, 2022, 4(3): 63-74.

2024年第5卷第4期

预览PDF | 下载PDF

引用本文

BibTeX

文章信息

doi: 10.12398/j.issn.2096-7217.2024.04.004

接收时间：2023-10-25
首发时间：2026-11-05
出版时间：2024-11-15

补充材料

相关文章

文章信息

作者

出版历史

收稿日期：2023-10-25
修回日期：2023-12-29

基金

National Key Research and Development Program of China(2022YFD2202103)

National Natural Science Foundation of China(31971798)

Zhejiang Provincial Key Research & Development Plan(2023C02049; 2023C02053)

SNJF Science and Technology Collaborative Program of Zhejiang Province(2022SNJF017)

Hangzhou Agricultural and Social Development Research Project(202203A03)

作者信息

^1.浙江理工大学机械工程学院，浙江杭州，310018

^2.浙江省种植装备技术重点实验室，浙江杭州，310018

^3.农业农村部东南丘陵山区农业装备重点实验室（部省共建），浙江杭州，310018

通讯作者:

参考文献

分享链接

https://castjournals.cast.org.cn/joweb/znhnyzbxb/CN/10.12398/j.issn.2096-7217.2024.04.004

分享至

全文二维码

扫描看全文

引用本文

BibTeX

本文的引用情况

2种不同金属材料的力学参数

科 Family	属数 Number of genus	种数 Number of species	占总种数比例 Percentage of total species (%)	属 Genus	种数 Number of species	占总种数比例 Percentage of total species (%)
鹅膏菌科Amanitaceae	2	11	5.26	鹅膏菌属 Amanita	10	4.78
小菇科 Mycenaceae	2	12	5.74	丝盖伞属 Inocybe	5	2.39
多孔菌科 Polyporaceae	8	14	6.70	蜡蘑属 Laccaria	5	2.39
红菇科 Russulaceae	3	23	11.00	小皮伞属 Marasmius	6	2.87
				小菇属 Mycena	11	5.26
				光柄菇属 Pluteus	5	2.39
				红菇属 Russula	17	8.13
				栓菌属 Trametes	5	2.39

关闭全屏

BibTeX
EndNote
RefWorks
TxT