arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.05982cs.ROcs.CV
收藏

HarvestFlex: 通过视觉-语言-动作策略适应实现草莓采摘

HarvestFlex: Strawberry Harvesting via Vision-Language-Action Policy Adaptation in the Wild

The Intelligent Equipment Research Center, Beijing Academy of Agriculture and Forestry Sciences(北京农业与林业科学研究院智能装备研究所);The School of Mechanical Electrical Engineering and Automation, Shanghai University(上海大学机械电子工程与自动化学院)

Ziyang Zhao, Shuheng Wang, Zhonghua Miao, Ya Xiong

AI总结:

HarvestFlex通过视觉-语言-动作策略适应,在真实温室中实现了高效草莓采摘,成功率达74%。

摘要:

本工作首次研究了将视觉-语言-动作(VLA)策略转移到真实温室桌面草莓采摘任务中的转移问题,这是一个长周期、无结构的任务,受到遮挡和镜面反射的挑战。我们在HarvestFlex平台上构建了一个端到端的闭环系统,使用三视角RGB传感(两个固定场景视角加一个腕部安装的视角),并有意避免深度云和显式几何校准。我们收集了3.71小时的VR遥控演示(227个回合),并使用全微调和LoRA对pi_0、pi_0.5和WALL-OSS进行了微调。在统一的50次真实温室协议和涵盖完成度、pi_0.5全微调在统一的50次真实温室协议和涵盖完成度、pi_0.5全微调实现了74.0%的成功率,32.6秒/次采摘,损伤率为4.1%。异步推理-控制解耦进一步在同步部署上提高了性能。结果表明,非平凡的闭环采摘在少于四小时的真实数据下实现,但仍受近距离可观察性损失和接触动力学不匹配的限制。演示视频可在:https://youtu.be/bN8ZowZKPMI获取。

英文摘要:

This work presents the first study on transferring vision-language-action (VLA) policies to real greenhouse tabletop strawberry harvesting, a long-horizon, unstructured task challenged by occlusion and specular reflections. We built an end-to-end closed-loop system on the HarvestFlex platform using three-view RGB sensing (two fixed scene views plus a wrist-mounted view) and intentionally avoided depth clouds and explicit geometric calibration. We collected 3.71 h of VR teleoperated demonstrations (227 episodes) and fine-tuned pi_0, pi_0.5, and WALL-OSS with full fine-tuning and LoRA. Under a unified 50 trials real-greenhouse protocol and metrics spanning completion, pi_0.5 with full fine-tuning achieved success rate of 74.0% with 32.6 s/pick and damage rate of 4.1%. Asynchronous inference-control decoupling further improved performance over synchronous deployment. Results showed non-trivial closed-loop picking with fewer than four hours of real data, while remaining limited by close-range observability loss and contact-dynamics mismatch. A demonstration video is available at: https://youtu.be/bN8ZowZKPMI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑