Driven by applications in autonomous driving robotics and augmented reality 3D object annotation presents challenges beyond 2D annotation including spatial complexity occlusion and viewpoint inconsistency Existing approaches based on single models often struggle to address these issues effectively We propose Tri MARF a novel framework that integrates tri modal inputs including 2D multi view images textual descriptions and 3D point clouds within a multi agent collaborative architecture to enhance large scale 3D annotation Tri MARF consists of three specialized agents a vision language model agent for generating multi view descriptions an information aggregation agent for selecting optimal descriptions and a gating agent that aligns textual semantics with 3D geometry for refined captioning Extensive experiments on Objaverse LVIS Objaverse XL and ABO demonstrate that Tri MARF substantially outperforms existing methods achieving a CLIPScore of 88 point 7 compared to prior state of the art methods retrieval accuracy of 45 point 2 and 43 point 8 on ViLT R at 5 and a throughput of up to 12000 objects per hour on a single NVIDIA A100 GPU
DeepMEL: A Multi-Agent Collaboration Framework for Multimodal Entity Linking
School of Computer Science, Peking University(北京大学计算机科学学院);School of Information Science and Engineering, Chongqing Jiaotong University(重庆交通大学信息科学与工程学院);China Research and Development Academy of Machinery Equipment(机械电子研究发展院);Center of Information Research, Academy of Military Science(军事科学信息研究中心)
VILLAIN at AVerImaTeC: Verifying Image-Text Claims via Multi-Agent Collaboration
School of AI Convergence, Soongsil University(人工智能融合学院,顺世大学);MAUM AI Inc.(MAUM人工智能公司);Department of Intelligent Semiconductors, Soongsil University(智能半导体系,顺世大学)
Cross-modal Affinity-aligned Multimodal Learning Analytics for Predicting Student Collaboration Satisfaction in Game-Based Learning
Institute of Education, National Cheng Kung University(国立成功大学教育研究所);Institute of Intelligent System, National Yang Ming Chiao Tung University(阳明交通大学智能系统研究所);Department of Computer Science, University at Albany, State University of New York(纽约州立大学水牛城分校计算机科学系)
Dense Audio-Visual Event Localization under Cross-Modal Consistency and Multi-Temporal Granularity Collaboration
Hefei University of Technology(合肥工业大学);University of Science and Technology of China(中国科学技术大学);Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Residual Optimal Transport-Based Experts Collaboration Towards Modality-Aware Infrared-Visible Object Detection
Xidian University(西安电子科技大学);Nanyang Technological University(南洋理工大学);Zhengzhou University(郑州大学);China University of Geosciences(中国地质大学);Swinburne University of Technology(斯威本科技大学)
Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration
Department of Computer Science, University of Kaiserslautern-Landau (RPTU)(科斯拉尔特伦大学计算机科学系);SDS, German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));INRAE, UMR TETIS, University of Montpellier(蒙彼利埃大学UMR TETIS);CIRAD, UMR TETIS, University of Montpellier(蒙彼利埃大学UMR TETIS);INRIA, EVERGREEN, University of Montpellier(蒙彼利埃大学)
MUSES: 3D-Controllable Image Generation via Multi-Modal Agent Collaboration
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学)
A Distributed Multi-Modal Sensing Approach for Human Activity Recognition in Real-Time Human-Robot Collaboration
University of Genoa(热那亚大学);The University of Danang–University of Science and Technology(丹江-科学技术大学);Japan Advanced Institute of Science and Technology(日本先进科学研究院)
Multimodal Collaboration Networks for Geospatial Vehicle Detection in Dense, Occluded, and Large-Scale Events
Beijing University of Posts and Telecommunications(北京邮电大学);Fuzhou University(福州大学);Univ. Grenoble Alpes(格勒诺布尔大学);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔综合理工学院);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);German Aerospace Center(德国航空航天中心)
Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams
Nanjing University of Science and Technology(南京理工大学);Baidu Inc(百度公司);AIML, Adelaide University(阿德莱德大学AIML);SUTD(新加坡科技设计大学);Southeast University(东南大学);East China Normal University(华东师范大学);University of Oxford(牛津大学)
Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology(电子与计算机工程系,香港科学与技术大学);Department of Computer Science and Engineering, The Hong Kong University of Science and Technology(计算机科学与工程系,香港科学与技术大学)
MCL-AD: Multimodal Collaboration Learning for Zero-Shot 3D Anomaly Detection
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan)(计算能力网络与信息安全部教育部重点实验室,山东计算机科学中心(济南国家超级计算机中心));Qilu University of Technology (Shandong Academy of Sciences)(齐鲁工业大学(山东科学院));Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science(山东省计算能力互联网与服务计算重点实验室,山东省计算机科学基础研究中心)
Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification
GFZ Helmholtz Center for Geosciences(德国波茨坦地学研究中心亥姆霍兹中心);INRAE, UMR TETIS, University of Montpellier(法国蒙彼利埃大学农业环境与生态研究院、蒙彼利埃大学TETIS联合研究单位);INRIA, EVERGREEN, University of Montpellier(法国蒙彼利埃大学信息与自动化研究所、EVERGREEN实验室);CIRAD, UMR TETIS, University of Montpellier(法国蒙彼利埃大学国际农业研究磋商组织、蒙彼利埃大学TETIS联合研究单位)
TMT: Tri-Modal Translation between Speech, Image, and Text by Processing Different Modalities as Different Languages
Integrated Vision and Language Lab., School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)(整合视觉与语言实验室,电气工程学院,韩国科学技术院(KAIST));Language Technologies Institute, Carnegie Mellon University(语言技术研究所,卡内基梅隆大学)
Hollywood Town: Long-Video Generation via Cross-Modal Multi-Agent Orchestration
The Hong Kong University of Science and Technology(香港科学与技术大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));University of Chicago(芝加哥大学);Stanford University(斯坦福大学)
Dissecting RGB-D Learning for Improved Multi-modal Fusion
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及跨学科应用国家重点实验室(东南大学));Tsinghua University(清华大学);College of Computer Science, Beijing University of Technology(北京理工大学计算机学院)
TriPSS: A Tri-Modal Keyframe Extraction Framework Using Perceptual, Structural, and Semantic Representations
Computer and Information Science, University of Arkansas - Little Rock(计算机与信息科学,亚拉荷马州立大学);ICSI, University of California, Berkeley(ICSI,加州大学伯克利分校);COSMOS Research Center, University of Arkansas - Little Rock(COSMOS研究中心,亚拉荷马州立大学)
2D or 3D: Who Governs Salience in VLA Models? -- Tri-Stage Token Pruning Framework with Modality Salience Awareness
School of Computer Science, Peking University(北京大学计算机科学学院);ZTE Corporation(中兴通讯);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院);School of Computer Science, China University of Geosciences (Wuhan)(中国地质大学(武汉)计算机科学学院);School of Electronics Engineering and Computer Science, Peking University(北京大学电子工程与计算机科学学院)