动态环境中具身智能体的测试时间世界模型混合
Test-Time Mixture of World Models for Embodied Agents in Dynamic Environments
AI总结：
TMoW通过测试时间更新路由函数，提升具身智能体在动态环境中的适应性与持续学习能力。
摘要：
基于语言模型（LM）的具身智能体正越来越多地应用于现实世界。然而，它们在动态环境中适应性仍然有限，其中构建准确且灵活的世界模型对于有效的推理和决策至关重要。为了解决这一挑战，我们扩展了混合专家（MoE）范式到具身智能体。虽然传统MoE架构将知识模块化为专家组件，并通过预训练路由进行知识分配，但一旦部署就变得僵化，使其在动态环境中适应未见领域时效果较差。因此，我们提出测试时间世界模型混合（TMoW），一种增强对未见和演变领域的适应性的框架。TMoW在测试时间更新其对世界模型的路由函数，不同于传统MoE中固定不变的函数，使智能体能够重新组合现有模型并整合新模型以实现持续适应。它通过（i）多粒度原型基于路由，适应从物体到场景级别的相似性，（ii）测试时间细化，使未见领域的特征在推理期间与原型对齐，以及（iii）蒸馏混合基于增强，高效地从少量数据和现有原型中构建新模型。我们在VirtualHome、ALFWorld和RLBench基准上评估TMoW，证明了其在零样本适应和少量样本扩展场景中的强大性能，并显示其使具身智能体能够在动态环境中有效运作。
英文摘要：
Language model (LM)-based embodied agents are increasingly deployed in real-world settings. Yet, their adaptability remains limited in dynamic environments, where constructing accurate and flexible world models is crucial for effective reasoning and decision-making. To address this challenge, we extend the Mixture-of-Experts (MoE) paradigm to embodied agents. While conventional MoE architectures modularize knowledge into expert components with pre-trained routing, they remain rigid once deployed, making them less effective for adapting to unseen domains in dynamic environments. We therefore propose Test-time Mixture of World Models (TMoW), a framework that enhances adaptability to unseen and evolving domains. TMoW updates its routing function over world models at test time, unlike conventional MoE where the function remains fixed, enabling agents to recombine existing models and integrate new ones for continual adaptation. It achieves this through (i) multi-granular prototype-based routing, which adapts mixtures across object- to scene-level similarities, (ii) test-time refinement that aligns unseen domain features with prototypes during inference, and (iii) distilled mixture-based augmentation, which efficiently constructs new models from few-shot data and existing prototypes. We evaluate TMoW on VirtualHome, ALFWorld, and RLBench benchmarks, demonstrating strong performance in both zero-shot adaptation and few-shot expansion scenarios, and showing that it enables embodied agents to operate effectively in dynamic environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
