发表机构
College of Computer and Information Sciences, Fujian Agriculture and Forestry University; School of Mechatronic Engineering and Automation, Shanghai University; Graduate School of Engineering Science, The University of Osaka（福建农林大学计算机与信息学院; 上海大学机电工程与自动化学院; 大阪大学工程科学研究生院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
HarvestPoint-ACT通过显式感知目标与采摘点，结合调度器选择目标，在树冠模型上实现88%采摘成功率，重度遮挡下达75%，解决机器人水果采摘的遮挡与闭合点推断问题。
AI 中文摘要
端到端模仿学习可避免手动设计机器人的接近与抓取动作，但策略仍需决定采摘哪个水果以及在何处闭合夹爪。遮挡会使策略在采摘过程中丢失已选水果，且仅从像素难以推断正确的闭合点。本文提出HarvestPoint-ACT，该方法在感知阶段明确做出这两项决策并将其提供给策略。带有关键点分支的实例分割前端为每个可见水果预测掩码和采摘点，其中采摘点指定夹爪闭合的位置。调度器根据遮挡情况和移动距离对检测到的候选水果进行排名并选择一个目标，每次尝试后会重新检测并重新排名候选水果，因为树冠可能已发生变化。所选水果被编码为8维状态，输入到动作分块Transformer中，该状态包含绝对采摘点、从夹爪到该点的向量、有效性标志和置信度分数。当所选水果暂时未被检测到时，系统会保留机器人基座坐标系中最后一次的采摘点估计值并将其标记为失效，若持续丢失则中止本次尝试。在树冠模型上，HarvestPoint-ACT的成功率达88%，在重度遮挡下的成功率为75%。
英文摘要
End-to-end imitation learning avoids hand-made robot motion for approaching and grasping, but the policy must still decide which fruit to pick and where to close the gripper. Occlusion can make the policy lose the selected fruit during harvesting, and the correct closing point is difficult to infer from pixels alone. This paper presents HarvestPoint-ACT, which makes both decisions explicit in perception and provides them to the policy. An instance segmentation front end with a keypoint branch predicts a mask and a harvest point for each visible fruit, where the harvest point specifies the location to close the gripper. A scheduler ranks detected candidates by occlusion and travel distance and selects one target. After each attempt, it redetects and reranks the candidates because the canopy may have changed. The selected fruit is encoded for an action chunking transformer as an eight-dimensional state, containing the absolute harvest point, the vector from the gripper to that point, a validity flag, and a confidence score. When the selected fruit is temporarily undetected, the system retains the last harvest point estimate in the robot base frame and marks it as stale, and aborts the attempt if the loss persists. On a canopy mock-up, HarvestPoint-ACT achieves a success rate of 88%, and of 75% under heavy occlusion.
CommentsSubmit to IEEE ROBIO 2026
