iTeach：真实环境中的交互教学用于机器人感知的故障驱动适应
iTeach: In the Wild Interactive Teaching for Failure-Driven Adaptation of Robot Perception
AI总结：
iTeach通过故障驱动的交互教学框架，实现机器人感知在真实环境中的自适应。该方法通过人类与物体的短时交互暴露信息配置，结合少量样本进行迭代微调，提升分割性能和抓取成功率。
摘要：
机器人感知模型在真实环境中部署时，常因分布外条件如遮挡、新物体实例等而失效。现有方法通过离线数据收集和再训练解决此问题，但效率低且无法在部署时修复故障。我们提出iTeach，一种故障驱动的交互教学框架，用于适应机器人感知。在部署过程中，人类观察模型预测，识别故障案例，并通过短时人类-物体交互（HumanPlay）暴露信息配置，同时记录RGB-D序列。为减少标注工作量，iTeach采用少样本半监督（FS3）标注策略，仅标注交互序列的最后帧，利用无手眼注视和语音命令进行标注，并将标签传播到视频中以生成密集监督。收集的故障驱动样本用于迭代微调，使感知模型在部署时逐步适应。我们在未见物体实例分割（UOIS）任务上评估iTeach，从预训练的MSMFormer模型开始。使用少量故障驱动样本，我们的方法在多样化的现实场景中显著提升了分割性能。这些改进直接转化为SceneReplica基准和真实机器人实验中更高的抓取和拾取成功率。我们的结果表明，故障驱动的协同交互教学能够高效地实现机器人感知在真实环境中的适应，并提升下游操作性能。项目页面：https://irvlutd.github.io/iTeach
英文摘要：
We present iTeach, a deployable system that lets any co-located human fix a robot's perception failures on the spot without expertise, a workstation, or offline retraining. The operator wears a mixed reality (MR) headset, sees the robot's segmentation predictions overlaid on the real scene, and corrects failures hands-free: rearranging objects (HumanPlay), annotating via gaze and voice, and triggering SAM2 backward mask propagation. Each ~20 s interaction yields 150-300 densely labeled training frames; the system fine-tunes the perception model onboard, keeps the better model, and redeploys, all without leaving the deployment site. The full loop requires only an RGB-D camera, onboard GPU, and an MR headset: any mobile robot, any environment. Starting from 26.1 on cluttered real-world scenes, 45 teaching interactions (13K frames) raise segmentation to 80.7 with no catastrophic forgetting; on three standard benchmarks the model never trained on, performance improves as well. Downstream pick-and-place on SceneReplica reaches 72/100, surpassing a model-based pipeline requiring CAD models. A 12-participant user study confirms non-experts match experts on annotation accuracy (~95% box IoU), speed, and task load (NASA-TLX 21/100). The framework is architecture-agnostic: any fine-tunable perception model can serve as backbone.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
