诊断序列机器人任务中的组合泛化问题
Diagnosing Compositional Generalization in Sequential Robot Tasks
- University of California, Berkeley（加州大学伯克利分校）
- Tsinghua University（清华大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文针对序列机器人操作的组合泛化问题，将泛化差距分解为三类偏移，发现结构化子集覆盖动作相关依赖即可实现高分布外性能，微调单演示可大幅提升OOD成功率，提出数据收集应优先依赖覆盖。
AI中文摘要：
序列机器人操作要求策略能执行熟悉指令组件的新颖组合。然而，为所有可能的指令元组收集演示会产生组合成本，而覆盖稀疏的数据集在分布外重组时往往失效。本文从指令空间覆盖的角度研究组合泛化，将泛化差距分解为三个来源：边缘指令偏移、指令组合偏移和上下文-动作偏移。该分解使我们能够诊断稀疏训练覆盖何时足够，以及训练集必须保留何种结构以实现可靠的动作预测。我们的结果表明，无需穷举元组枚举：当结构化子集覆盖与动作相关的依赖关系时，其规模仅为完整任务空间的四分之一即可恢复强大的分布外性能。我们进一步发现，稀疏训练失效通常是由于指令引导而非缺少低级技能；每个任务仅微调一个演示，可将分布外成功率从0.4%提升至54.7%。对于语义依赖任务，有效覆盖必须捕捉关系结构，而非仅因子多样性。这些发现表明，高效的机器人数据收集应优先考虑指令空间中的依赖覆盖，而非穷举任务扩展。更多结果见补充材料，项目网站为this https URL。
英文摘要：
Sequential robot manipulation requires policies to execute novel combinations of familiar instruction components. However, collecting demonstrations for all possible instruction tuples is combinatorially expensive, while sparsely covered datasets often fail under out-of-distribution recombination. This paper studies compositional generalization through the lens of instruction-space coverage. We decompose the generalization gap into three sources: \textit{marginal instruction shift}, \textit{instruction-compositional shift}, and \textit{context--action shift}. This decomposition allows us to diagnose when sparse training coverage is sufficient, and what structure the training set must preserve for reliable action prediction. Our results show that exhaustive tuple enumeration is unnecessary: a structured subset, as small as one quarter of the full task space, can recover strong out-of-distribution performance when it covers action-relevant dependencies. We further find that sparse training often fails due to instruction steering rather than missing low-level skills; finetuning only one demonstration per task improves OOD success from \(0.4\%\) to \(54.7\%\). For semantically dependent tasks, effective coverage must capture relational structure rather than only factor diversity. These findings suggest that efficient robot data collection should prioritize dependency coverage in instruction space over exhaustive task expansion. More results are available in the supplementary material. Project website: https://yixiaowang7.github.io/Diagnosing_Compositional_Generalization_Robot_Page/.
