arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.08748cs.ROcs.AI

面向机器人操作的视觉提示:采用带标注引导的拾取与放置方法,基于ACT算法

Visual Prompting for Robotic Manipulation with Annotation-Guided Pick-and-Place Using ACT

Embodied AI Research Team(具身人工智能研究团队);National Institute of AIST(国家信息与系统技术研究所)

Muhammad A. Muttaqien, Tomohiro Motoda, Ryo Hanai, Yukiyasu Domae

AI总结:

针对便利店机器人拾取放置任务的挑战,提出带标注引导视觉提示的感知-行动流水线,采用ACT算法实现自适应操作,提升了零售环境下的抓取精度与适应性。

摘要:

便利店中的机器人拾取与放置任务面临诸多挑战,包括物体排列密集、存在遮挡,以及物体在颜色、形状、尺寸、纹理等属性上的差异,这些因素使轨迹规划与抓取变得复杂。本文提出一种利用标注引导的视觉提示的感知-行动流水线,其中边界框标注可识别可拾取物体与放置位置,提供结构化空间引导。与传统的分步规划不同,本文采用基于Transformer的动作分块算法(ACT)作为模仿学习算法,使机械臂能从人类演示中预测分块动作序列,实现流畅、自适应且数据驱动的拾取与放置操作。本文基于成功率和抓取行为的视觉分析对系统进行评估,结果显示在零售环境中抓取精度与适应性得到提升。

英文摘要:

Robotic pick-and-place tasks in convenience stores pose challenges due to dense object arrangements, occlusions, and variations in object properties such as color, shape, size, and texture. These factors complicate trajectory planning and grasping. This paper introduces a perception-action pipeline leveraging annotation-guided visual prompting, where bounding box annotations identify both pickable objects and placement locations, providing structured spatial guidance. Instead of traditional step-by-step planning, we employ Action Chunking with Transformers (ACT) as an imitation learning algorithm, enabling the robotic arm to predict chunked action sequences from human demonstrations. This facilitates smooth, adaptive, and data-driven pick-and-place operations. We evaluate our system based on success rate and visual analysis of grasping behavior, demonstrating improved grasp accuracy and adaptability in retail environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑