发表机构
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出神经符号架构，在模拟玩具卡车组装领域，借助具身对话与任务演示，通过自然语言传递语义约束提升智能体在线适应的数据效率，解决部署后遇未知语义约束的组装问题。
AI 中文摘要
本文提出一种神经符号架构，用于借助具身对话与任务演示的证据学习组装新型结构。我们聚焦于智能体部署后遇到结构语义约束的场景——即训练时未提供的、关于哪些部件类型与特征可构成有效结构的约束，且智能体初始时并不知晓相关结构与部件概念。智能体必须在尝试组装的过程中，通过用户交互获取并利用此类知识。我们在模拟玩具卡车组装领域研究该场景，从自然语言编码的符号证据与密集视觉观测中学习。实验表明，通过自然语言传递语义约束（如“自卸卡车有一个自卸斗”），相比仅依赖任务演示或仅通过自然语言命名部件，能实现数据效率更高的在线适应。
英文摘要
This paper describes a neurosymbolic architecture for learning to assemble novel structures using evidence from embodied conversations and task demonstrations. We focus on scenarios where an agent encounters, after deployment, semantic constraints on structures--in other words, constraints as to which part types and features make valid structures--that were not available during training, and where it is initially unaware of the relevant structure and component part concepts. The agent must acquire and exploit such knowledge through user interactions while attempting assembly. We study this setting in a simulated toy truck assembly domain, learning from symbolic evidence encoded in natural language and from dense visual observations. Our experiments show that communicating semantic constraints through natural language (e.g., "dump trucks have a dumper") yields more data-efficient online adaptation than relying only on task demonstrations and/or only naming the parts through natural language.
CommentsAccepted to, and to appear in the 20th Conference on Neurosymbolic Learning and Reasoning (NeSy 2026)
