A.D.A.M.O.（具多模态观测的语言驱动行动智能体）：面向虚拟人类的视觉-符号框架
A.D.A.M.O. (Agent for language-Driven Actions with Multimodal Observations): A Visual-Symbolic Framework for Virtual Humans
查看机构详情
- DAUIN Politecnico di Torino（都灵理工大学 DAUIN）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
提出A.D.A.M.O.视觉-符号框架，利用预训练VLM与工具调用统一感知、推理和行动，通过双世界模型和CD分类任务套件，实验表明语义标注显著影响任务完成与失败模式。
中文摘要 AI 辅助
创建可信的虚拟人类需要将感知、推理和行动通过语言进行连贯整合。核心挑战在于将这些组件组合成一个基于交互式3D环境的控制回路。为此，我们提出了A.D.A.M.O.（具多模态观测的语言驱动行动智能体），一个面向语言驱动虚拟人类的视觉-符号框架，它利用预训练的视觉语言模型（VLM）与工具调用功能，将感知、推理和行动统一在单一控制回路中。A.D.A.M.O.维护一个双重视觉-符号世界模型，该模型结合了以自我为中心的视觉输入和同步的符号状态，以支持从自然语言提示出发的、基于目标的面向任务行为。为了支持诊断性评估，我们引入了一个受控任务套件，该套件由认知难度（CD）分类法组织，将空间任务分解为程序性复杂度和语言复杂度。在受控场景中的实验表明，语义标注强烈影响任务完成率和失败模式，减少了感知模糊性，同时将失败转移到下游执行阶段，而推理错误则相对较少见。
英文摘要
Creating believable vh requires the coherent integration of perception, reasoning, and action mediated by language. A central challenge is to combine these components into a control loop grounded in interactive 3D environments. To this end, we present A.D.A.M.O. (Agent for language-Driven Actions with Multimodal Observations), a visual-symbolic framework for language-driven vh that leverages a pretrained vlm with tool calling to unify perception, reasoning, and action within a single control loop. A.D.A.M.O. maintains a dual visual-symbolic world model that combines egocentric visual input and synchronized symbolic state to support grounded task-oriented behavior from natural language prompts. To support diagnostic evaluation, we introduce a controlled task suite organized by a cd taxonomy that breaks down spatial tasks into procedural and linguistic complexity. Experiments in controlled scenes show that semantic labeling strongly influences task completion and failure modes, reducing perceptual ambiguity while shifting failures toward downstream execution, whereas reasoning errors remain comparatively rare.
