多模态大模型

Online,Target-Free LiDAR-Camera Extrinsic Calibration via Cross-Modal Mask Matching

Zhiwei Huang, Yikang Zhang, Qijun Chen, Rui Fan

专题命中 多模态Agent ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments accepted to IEEE Trans. on Intelligent Vehicles (T-IV)

EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning

Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, Xihui Liu

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Project released at: https://github.com/ChenYi99/EgoPlan

WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models

Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, Dong Yu

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to ACL 2024 (main). Code and data is released at https://github.com/MinorJerry/WebVoyager

You Only Look at Screens: Multimodal Chain-of-Action Agents

Zhuosheng Zhang, Aston Zhang

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Findings of ACL 2024

Autonomous Workflow for Multimodal Fine-Grained Training Assistants Towards Mixed Reality

Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Jiang Yiming, Yao Sai, Di Wang, Zhumin Chen, Pengjie Ren, Pablo Cesar

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACL 2024

MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs

Xianhao Yu, Jiaqi Fu, Renjia Deng, Wenjuan Han

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Project website: https://github.com/cocacola-lab/MineLand

Influence based explainability of brain tumors segmentation in multimodal Magnetic Resonance Imaging

Tommaso Torda, Andrea Ciardiello, Simona Gargiulo, Greta Grillo, Simone Scardapane, Cecilia Voena, Stefano Giagu

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 15 pages, 7 figures

Multimodal Large Language Models to Support Real-World Fact-Checking

Jiahui Geng, Yova Kementchedjhieva, Preslav Nakov, Iryna Gurevych

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedback

Dong Won Lee, Hae Won Park, Yoon Kim, Cynthia Breazeal, Louis-Philippe Morency

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 10 pages, 3 figures, 2 tables

Restoring Ancient Ideograph: A Multimodal Multitask Neural Network Approach

Siyu Duan, Jun Wang, Qi Su

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accept by Lrec-Coling 2024

META-GUI: Towards Multi-modal Conversational Agents on Mobile GUI

Liangtai Sun, Xingyu Chen, Lu Chen, Tianle Dai, Zichen Zhu, Kai Yu

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 10 figures

LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model

Yichen Zhu, Minjie Zhu, Ning Liu, Zhicai Ou, Xiaofeng Mou, Jian Tang

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments The datasets were incomplete as they did not include all the necessary copyrights

RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agents

Tomoyuki Kagaya, Thong Jing Yuan, Yuxuan Lou, Jayashree Karlekar, Sugiri Pranata, Akira Kinose, Koki Oguri, Felix Wick, Yang You

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

3M-TRANSFORMER: A Multi-Stage Multi-Stream Multimodal Transformer for Embodied Turn-Taking Prediction

Mehdi Fatan, Emanuele Mincato, Dimitra Pintzou, Mariella Dimiccoli

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to ICASSP 2024

Modality Plug-and-Play: Elastic Modality Adaptation in Multimodal LLMs for Embodied AI

Kai Huang, Boyuan Yang, Wei Gao

专题命中 多模态Agent ：multimodal(title)；cross-modal(abstract)；分类 cs.CL、cs.AI

GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation

An Yan, Zhengyuan Yang, Wanrong Zhu, Kevin Lin, Linjie Li, Jianfeng Wang, Jianwei Yang, Yiwu Zhong, Julian McAuley, Jianfeng Gao, Zicheng Liu, Lijuan Wang

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Work in progress

Multitask Multimodal Prompted Training for Interactive Embodied Task Completion

Georgios Pantazopoulos, Malvina Nikandrou, Amit Parekh, Bhathiya Hemanthage, Arash Eshghi, Ioannis Konstas, Verena Rieser, Oliver Lemon, Alessandro Suglia

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments EMNLP 2023

Guide Your Agent with Adaptive Multimodal Rewards

Changyeon Kim, Younggyo Seo, Hao Liu, Lisa Lee, Jinwoo Shin, Honglak Lee, Kimin Lee

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2023. Project webpage: https://sites.google.com/view/2023arp

MAEA: Multimodal Attribution for Embodied AI

Vidhi Jain, Jayant Sravan Tamarapalli, Sahiti Yerramilli, Yonatan Bisk

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Contextualized Plan Prediction for Embodied Task Completion

Mert İnan, Aishwarya Padmakumar, Spandana Gella, Patrick Lange, Dilek Hakkani-Tur

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments NILLI at EMNLP 2022

Multi-modal Cooking Workflow Construction for Food Recipes

Liangming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, Yu-Gang Jiang, Tat-Seng Chua

专题命中 多模态Agent ：multi-modal(title,abstract)；分类 cs.CL、cs.MM

Comments This manuscript has been accepted at ACM MM 2020

Embodied Multimodal Multitask Learning

Devendra Singh Chaplot, Lisa Lee, Ruslan Salakhutdinov, Devi Parikh, Dhruv Batra

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments See https://devendrachaplot.github.io/projects/EMML for demo videos

A Knowledge-Grounded Multimodal Search-Based Conversational Agent

Shubham Agarwal, Ondrej Dusek, Ioannis Konstas, Verena Rieser

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI, pages 59-66, Brussels, Belgium, October 2018

Learning to play: A Multimodal Agent for 3D Game-Play

Yuguang Yue, Irakli Salia, Samuel Hunt, Christopher Green, Wenzhe Shi, Jonathan J Hunt

机构 * Player2 纠错

专题命中 多模态Agent ：multimodal(title)；multi-modal(abstract,comments)；分类 cs.AI

Comments International Conference on Computer Vision Workshop on Multi-Modal Reasoning for Agentic Intelligence

FairCVtest Demo: Understanding Bias in Multimodal Learning with a Testbed in Fair Automatic Recruitment

Alejandro Peña, Ignacio Serna, Aythami Morales, Julian Fierrez

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.CV

Comments ACM Intl. Conf. on Multimodal Interaction (ICMI). arXiv admin note: substantial text overlap with arXiv:2004.07173

VDC-Agent: When Video Detailed Captioners Evolve Themselves via Agentic Self-Reflection

VDC-Agent：当视频详细描述器通过代理自我反思而自我进化

Qiang Wang, Xinyuan Gao, Yuhang He, Jizhou Han, Jiangyang Li, SongLin Dong, Zhiheng Ma, Yihong Gong

机构 * Xi’an Jiaotong University（西安交通大学） ； Kuaishou Technology（快手科技） ； Shenzhen University of Advanced Technology（深圳先进技术大学） 纠错

专题命中 多模态Agent ：MLLM(abstract,abstract_cn)；multimodal(abstract)；分类 cs.CV、cs.AI、cs.MM

AI总结 VDC-Agent通过自我反思机制实现视频详细描述的自我进化，利用自动生成的(描述，评分)对提升描述准确性与评分表现。

Comments Accepted to ECCV 2026. Project Page: https://vdcagent.github.io

Learning to Select Visual In-Context Demonstrations

学习选择视觉上下文示例

Eugene Lee, Yu-Chi Lin, Jiajie Diao

机构 * University of Cincinnati（辛辛那提大学） ； University of California, Los Angeles（加利福尼亚大学洛杉矶分校） 纠错

专题命中 多模态Agent ：MLLM(abstract,abstract_cn)；multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 本文提出LSD方法，通过强化学习构建最优演示集，提升多模态大语言模型在视觉回归任务中的表现，揭示了学习选择在视觉上下文学习中的必要性。

Comments 21 pages, 12 figure, accepted to Computer Vision and Pattern Recognition Conference (CVPR) 2026 Findings Track

Journal ref In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9455-9465) 2026

Verifiable Geometry Problem Solving: Solver-Driven Autoformalization and Theorem Proposing

可验证几何问题求解：求解器驱动的自动形式化与定理提出

Can Li, Ting Zhang, Junbo Zhao, Hua Huang

机构 * Beijing Normal University（北京师范大学） 纠错

专题命中 多模态Agent ：multimodal(abstract)；MLLM(abstract)；cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

AI总结 提出SD-GPS框架，通过求解器驱动的自动形式化和可验证定理提出，解决几何问题求解中神经符号方法的瓶颈，在Geometry3K和PGPS9K上超越现有方法。

HATS: A Human-Agent Teleoperation System for Multi-Arm Data Collection

HATS：用于多臂数据收集的人-智能体遥操作系统

Zesen Lin, Jian-Jian Jiang, Haoming Cen, Xiao-Ming Wu, Dandan Zhang, Wei-Shi Zheng

机构 * School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院） ； Nanyang Technological University（南洋理工大学） ； Imperial College London（帝国理工学院） 纠错

专题命中 多模态Agent ：MLLM(summary_cn,abstract)

AI总结 提出HATS系统，由单操作员借助MLLM智能体控制两主臂和两辅助臂，实现高效多臂数据收集，性能媲美双人专家团队。

Rule-Compliant Visual Spatial Planning for Multimodal Large Language Models

面向多模态大语言模型的符合规则的视觉空间规划

Yu Chen, Ting Lei, Yaoyi Li, Jia Cai, Zhecen Wu, Yang Liu

机构 * Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所） ； Yinwang Intelligent Technology Co., Ltd（银湾智能科技有限公司） 纠错

专题命中 多模态Agent ：multimodal(title,abstract)；分类 cs.AI

AI总结 该研究针对多模态大语言模型的规则遵循空间规划问题，构建了RuleMaze基准，提出语言-逻辑-函数混合方法和解耦多模态规划（DMP），提升了规则遵循度与规划成功率。