Citrus pose estimation from an RGB image for automated harvesting

文献类型： 外文期刊

第一作者： Sun, Qixin

作者： Sun, Qixin;Chai, Xiujuan;Zhou, Guomin;Sun, Qixin;Sun, Tan;Zhong, Ming;Yin, Hesheng;Zeng, Zhikang

作者机构：

关键词： Citrus; Convolutional neural network; 3D pose estimation; RGB image; Attentional multi-scale feature fusion

期刊名称：COMPUTERS AND ELECTRONICS IN AGRICULTURE （ 影响因子：8.3； 五年影响因子：8.3 ）

ISSN： 0168-1699

年卷期： 2023 年 211 卷

页码：

收录情况： SCI

摘要： Automated fruit harvesting is promising research in the development of agricultural modernization. However, the complex and non-structural orchard environment is extremely challenging. In order to meet the needs of different end-effectors and to improve the success rate of automatic fruit harvesting, it is critical to perform fruit pose estimation before picking operations. In this study, a citrus pose estimation method through a single RGB image is introduced. The rotation of the citrus pose is defined as a vector that passes through the center of the fruit, which is perpendicular to the plane where the fruit navel point is located. Simply speaking, a multi-task learning model named FPENet is proposed to simultaneously locate the fruit navel point and predict the fruit rotation vector. And a hyperparameter is introduced in the loss function to achieve the simultaneous convergence of multiple tasks. In addition, this paper designs a 2D image annotation tool and constructs a citrus pose dataset, which contributes to model training and also the algorithm evaluation. In the experiment, we evaluate and analyze each module of the proposed network structure, and verify its performance on a harvesting robot. The experimental results show that the FPENet achieves an 88.92 AP score on fruit navel point detection, and 11.13 degrees on the average error of the rotation vector. Over 90% of rotation vectors have an angular error of less than 22.5 degrees. The harvesting success rate is 79.79%. This study offers a new idea for fruit pose estimation and provides the possibility and foundation for estimating fruit pose with a 2D image input.

分类号：

  • 相关文献
作者其他论文 更多>>

微信小程序