Autonomous agents must learn to collaborate. It is not scalable to develop a new centralized agent every time a task's difficulty outpaces a single agent's abilities. While multi-agent collaboration research has flourished in gridworld-like environments, relatively little work has considered visually rich domains. Addressing this, we introduce the novel task FurnMove in which agents work together to move a piece of furniture through a living room to a goal. Unlike existing tasks, FurnMove requires agents to coordinate at every timestep. We identify two challenges when training agents to complete FurnMove: existing decentralized action sampling procedures do not permit expressive joint action policies and, in tasks requiring close coordination, the number of failed actions dominates successful actions. To confront these challenges we introduce SYNC-policies (synchronize your actions coherently) and CORDIAL (coordination loss). Using SYNC-policies and CORDIAL, our agents achieve a 58% completion rate on FurnMove, an impressive absolute gain of 25 percentage points over competitive decentralized baselines. Our dataset, code, and pretrained models are available at https://unnat.github.io/cordial-sync .
University of Washington（华盛顿大学）；Microsoft Research（微软研究院）
Embodied avatars as virtual agents have many applications and provide benefits over disembodied agents, allowing non-verbal social and interactional cues to be leveraged, in a similar manner to how humans interact with each other. We present an open embodied avatar built upon the Unreal Engine that can be controlled via a simple python programming interface. The avatar has lip syncing (phoneme control), head gesture and facial expression (using either facial action units or cardinal emotion categories) capabilities. We release code and models to illustrate how the avatar can be controlled like…
Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environments
Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center（新疆多模态智能处理与信息安全工程技术研究中心）；School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；School of Electrical Engineering and Automation, Tianjin University of Technology（天津理工大学电气工程与自动化学院）
ForceU-VLA: A Force-Aware Vision-Language-Action Model for Embodied Ultrasound Scanning
Faculty of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Information Science and Engineering, Shandong University（山东大学信息科学与工程学院）；Innovation School of Artificial Intelligence, Hefei University of Technology（合肥工业大学人工智能创新学院）
AirGroundBench: Probing Spatial Intelligence in Multimodal Large Models under Heterogeneous Multi-View Embodied Collaboration
Beijing University of Posts and Telecommunications（北京邮电大学）；QiYuanLab（启元实验室）；Tsinghua University（清华大学）；University of Electronic Science and Technology of China（电子科技大学）
VideoWeaver: Multimodal Multi-View Video-to-Video Transfer for Embodied Agents
Huawei Heisenberg Research Center（华为海森堡研究中心）；Ludwig Maximilian University of Munich（慕尼黑大学）；University of Freiburg（弗莱堡大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis
GigaAI（极佳科技）；Zhejiang University（浙江大学）；Peking University（北京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina
Faculty of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Information Science and Engineering, Shandong University（山东大学信息科学与工程学院）；the Qilu Second Hospital of Shandong University（山东大学第二齐鲁医院）；Innovation School of Artificial Intelligence, Hefei University of Technology（合肥工业大学人工智能创新学院）
