Large language models (LLMs) are increasingly grounded in sensor data to perceive and reason about human physiology and the physical world. However, accurately interpreting heterogeneous multimodal sensor data remains a fundamental challenge. We show that a single monolithic LLM often fails to reason coherently across modalities, leading to incomplete interpretations and prior-knowledge bias. We introduce ConSensus, a training-free multi-agent collaboration framework that decomposes multimodal sensing tasks into specialized, modality-aware agents. To aggregate agent-level interpretations, we propose a hybrid fusion mechanism that balances semantic aggregation, which enables cross-modal reasoning and contextual understanding, with statistical consensus, which provides robustness through agreement across modalities. While each approach has complementary failure modes, their combination enables reliable inference under sensor noise and missing data. We evaluate ConSensus on five diverse multimodal sensing benchmarks, demonstrating an average accuracy improvement of 7.1% over the single-agent baseline. Furthermore, ConSensus matches or exceeds the performance of iterative multi-agent debate methods while achieving a 12.7 times reduction in average fusion token cost through a single-round hybrid fusion protocol, yielding a robust and efficient solution for real-world multimodal sensing tasks. The source code is available at https://github.com/nokia/multi-agent-collaboration-for-multimodal-sensing.
Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaboration
Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究院(TeleAI),中国电信);Sichuan University(四川大学);Peking University(北京大学);Beijing Jiaotong University(北京交通大学);Harbin Institute of Technology(哈尔滨工业大学);China Railway Rolling Stock Corporation Limited(中国铁路滚动股票有限公司)
COMMA: A Communicative Multimodal Multi-Agent Benchmark
Department of Computer Sciences University of Wisconsin-Madison(计算机科学系威斯康星大学麦迪逊分校);Department of Computer Sciences UC San Diego(计算机科学系加州大学圣地亚哥分校);Department of Radiology University of Wisconsin-Madison(放射学系威斯康星大学麦迪逊分校);Department of Computer Sciences Department of Biostatistics and Medical Informatics University of Wisconsin-Madison(计算机科学系生物统计学与医学信息学系威斯康星大学麦迪逊分校)
DeepMEL: A Multi-Agent Collaboration Framework for Multimodal Entity Linking
School of Computer Science, Peking University(北京大学计算机科学学院);School of Information Science and Engineering, Chongqing Jiaotong University(重庆交通大学信息科学与工程学院);China Research and Development Academy of Machinery Equipment(机械电子研究发展院);Center of Information Research, Academy of Military Science(军事科学信息研究中心)
Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation
Graduate School of Information Science, University of Hyogo(京都大学垣田学园信息科学研究生院);Center for Informatics Science, School of Information Technology and Computer Science, Nile University(尼罗大学信息科学中心)
Disentangling Fact from Sentiment: A Dynamic Conflict-Consensus Framework for Multimodal Fake News Detection
Xinjiang University(新疆大学);AI Security Lab(360AI安全实验室);Beihang University(北航);South China University of Technology(华南理工大学);Fudan University(复旦大学);Shenzhen University(深圳大学)
SimWorld-Robotics: Synthesizing Photorealistic and Dynamic Urban Environments for Multimodal Robot Navigation and Collaboration
University of Virginia(弗吉尼亚大学);UC San Diego(加州大学圣地亚哥分校);Johns Hopkins University(约翰霍普金斯大学);Carnegie Mellon University(卡内基梅隆大学);University of Michigan(密歇根大学)
Pro-Router: Token-Aware Progressive Model Routing with Adaptive Edge-Cloud Collaboration for Efficient Multimodal LLM Inference
Anyscale(安尼斯科尔公司);Mississippi State University(密西西比州立大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Institute of AI for Industries, Chinese Academy of Sciences(中国科学院产业人工智能研究院)
Cross-modal Affinity-aligned Multimodal Learning Analytics for Predicting Student Collaboration Satisfaction in Game-Based Learning
Institute of Education, National Cheng Kung University(国立成功大学教育研究所);Institute of Intelligent System, National Yang Ming Chiao Tung University(阳明交通大学智能系统研究所);Department of Computer Science, University at Albany, State University of New York(纽约州立大学水牛城分校计算机科学系)
From Visual Perception to Deep Empathy: An Automated Assessment Framework for House-Tree-Person Drawings Using Multimodal LLMs and Multi-Agent Collaboration
Tsinghua University(清华大学);Shenzhen University(深圳大学);University of Oxford(牛津大学);Shenzhen Institute of Education Sciences(深圳市教育科学研究院);Guangzhou University of Chinese Medicine(广州中医药大学);Harbin Institute of Technology(哈尔滨工业大学)
Agora: Toward Autonomous Bug Detection in Production-Level Consensus Protocols with LLM Agents
School of Computing, National University of Singapore(新加坡国立大学计算机学院);School of Information and Telecommunication Engineering, Beijing University of Posts and Telecommunications(北京邮电大学信息与电信工程学院);Peking University(北京大学);G Labs(0G实验室)
Multi-Modal Manipulation via Multi-Modal Policy Consensus
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Columbia University(哥伦比亚大学);Massachusetts Institute of Technology(麻省理工学院);Harvard University(哈佛大学)
MedCoRAG: Interpretable Hepatology Diagnosis via Hybrid Evidence Retrieval and Multispecialty Consensus
School of Cyber Science and Engineering, Nanjing University of Science and Technology(南京理工大学信息科学与工程学院);School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Department of Neurology, China-Japan Union Hospital of Jilin University(吉林大学中日联谊医院神经内科);Department of Anesthesiology, China-Japan Union Hospital of Jilin University(吉林大学中日联谊医院麻醉科);School of Computing, Macquarie University(麦考瑞大学计算机学院)