看见并切换：基于视觉的交互式机器人技能编程分支方法

中文摘要 AI 辅助

编程通过示范（PbD）使机器人编程对非专业人士更加可访问，但目前的教学框架在处理现实世界变异性时仍面临挑战，尤其是在机器人必须从视觉输入中在线选择合适任务变体时。我们提出了See & Switch，一种交互式教学和执行框架，将任务表示为由决策状态连接的技能部分图，允许在回放过程中进行条件分支。其基于视觉的Switcher利用手眼一致图像选择合适的后继技能部分，并检测需要新示范的新型情况。该框架通过肢体教学、操纵杆控制和手势支持执行中的恢复示范。我们在三个灵活操作任务上对See & Switch进行了评估，使用8名新手用户，收集了约900个真实机器人执行回放。为了将视觉决策性能与决策状态中的定时错误分离，我们使用用户门控的决策状态窗口进行离线评估。在决策状态窗口内的评估中，该方法在分支选择准确率上达到最高90.6%，并在79个决策状态中的47个中以>90%的准确率检测异常，证明了基于视觉输入的可靠切换，用于条件机器人技能编程。我们提供所有代码和实验数据在http://imitrob.ciirc.cvut.cz/publications/seeandswitch。

英文摘要

Programming by demonstration (PbD) makes robot programming accessible to non-experts, but scaling it to real-world variability remains a challenge for current teaching frameworks, especially when a robot must select suitable task variants online from visual input. We present See & Switch, an interactive teaching-and-execution framework that represents tasks as graphs of skill parts connected by decision states, enabling conditional branching during replay. Its vision-based Switcher uses eye-in-hand images to select the appropriate successor skill part and detect novel situations that require new demonstrations. The framework supports recovery demonstrations during execution through kinesthetic teaching, joystick control, and hand gestures. We evaluate See & Switch on three dexterous manipulation tasks with 8 novice users, collecting approx. 900 real-robot execution rollouts. To isolate visual decision performance from timing errors during decision states, we evaluate the Switcher offline using user-gated decision state windows. In the evaluation within the decision state windows, the method achieves up to 90.6% branch-selection accuracy and detects anomalies with >90% accuracy in 47 of 79 decision states, demonstrating reliable switching based on visual input for conditional robot-skill programming. We provide all code and experiment data at http://imitrob.ciirc.cvut.cz/publications/seeandswitch.

补充信息

备注

8 pages, 9 figures