[预印本]HarvestPoint-ACT:遮挡条件下机器人水果采摘的显式目标选择与采摘点条件化(原标题:HarvestPoint-ACT: Explicit Target Selection and Harvest-Point Conditioning for Robotic Fruit Harvesting under Occlusion)
针对遮挡条件下机器人采摘的显式目标选择与采摘点条件化方法，具有较高技术价值，但属于细分领域进展，影响范围有限。
端到端模仿学习避开了手工制作接近抓取的机器人运动,但策略仍需决定采摘哪个水果以及在何处闭合夹爪。遮挡会使策略在采摘过程中丢失所选水果,正确的闭合点也难以仅从像素推断。本研究提出HarvestPoint-ACT,在感知中使两个决策显式化并提供给策略:实例分割前端带关键点分支为每个可见水果预测掩码和采摘点;调度器按遮挡和移动距离对检测候选排序并选择一个目标。每次尝试后重新检测和重新排序候选。所选水果被编码为8维状态,送入动作分块Transformer。在冠层模拟装置上成功率88%,重遮挡下75%。
Abstract:End-to-end imitation learning avoids hand-made robot motion for approaching and grasping, but the policy must still decide which fruit to pick and where to close the gripper. Occlusion can make the policy lose the selected fruit during harvesting, and the correct closing point is difficult to infer from pixels alone. This paper presents HarvestPoint-ACT, which makes both decisions explicit in perception and provides them to the policy. An instance segmentation front end with a keypoint branch predicts a mask and a harvest point for each visible fruit, where the harvest point specifies the location to close the gripper. A scheduler ranks detected candidates by occlusion and travel distance and selects one target. After each attempt, it redetects and reranks the candidates because the canopy may have changed. The selected fruit is encoded for an action chunking transformer as an eight-dimensional state, containing the absolute harvest point, the vector from the gripper to that point, a validity flag, and a confidence score. When the selected fruit is temporarily undetected, the system retains the last harvest point estimate in the robot base frame and marks it as stale, and aborts the attempt if the loss persists. On a canopy mock-up, HarvestPoint-ACT achieves a success rate of 88%, and of 75% under heavy occlusion.
Comments:Submit to IEEE ROBIO 2026
Subjects:Robotics (cs.RO)
Cite as:arXiv:2608.18446 [cs.RO]
(or arXiv:2608.18446v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2608.18446
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Hao Chen [view email]
[v1] Wed, 19 Aug 2026 02:21:27 UTC (7,450 KB)