多模态大模型

MLZero: A Multi-Agent System for End-to-end Machine Learning Automation

Haoyang Fang, Boran Han, Nick Erickson, Xiyuan Zhang, Su Zhou, Anirudh Dagar, Jiani Zhang, Ali Caner Turkmen, Cuixiong Hu, Huzefa Rangwala, Ying Nian Wu, Bernie Wang, George Karypis

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning

NVIDIA, :, Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, Liang Feng, Francesco Ferroni, Rama Govindaraju, Jinwei Gu, Siddharth Gururani, Imad El Hanafi, Zekun Hao, Jacob Huffman, Jingyi Jin, Brendan Johnson, Rizwan Khan, George Kurian, Elena Lantz, Nayeon Lee, Zhaoshuo Li, Xuan Li, Maosheng Liao, Tsung-Yi Lin, Yen-Chen Lin, Ming-Yu Liu, Xiangyu Lu, Alice Luo, Andrew Mathau, Yun Ni, Lindsey Pavao, Wei Ping, David W. Romero, Misha Smelyanskiy, Shuran Song, Lyne Tchapmi, Andrew Z. Wang, Boxin Wang, Haoxiang Wang, Fangyin Wei, Jiashu Xu, Yao Xu, Dinghao Yang, Xiaodong Yang, Zhuolin Yang, Jingxu Zhang, Xiaohui Zeng, Zhe Zhang

机构 * NVIDIA 纠错

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Seed1.5-VL Technical Report

Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, Jingji Chen, Jingjia Huang, Kang Lei, Liping Yuan, Lishu Luo, Pengfei Liu, Qinghao Ye, Rui Qian, Shen Yan, Shixiong Zhao, Shuai Peng, Shuangye Li, Sihang Yuan, Sijin Wu, Tianheng Cheng, Weiwei Liu, Wenqian Wang, Xianhan Zeng, Xiao Liu, Xiaobo Qin, Xiaohan Ding, Xiaojun Xiao, Xiaoying Zhang, Xuanwei Zhang, Xuehan Xiong, Yanghua Peng, Yangrui Chen, Yanwei Li, Yanxu Hu, Yi Lin, Yiyuan Hu, Yiyuan Zhang, Youbin Wu, Yu Li, Yudong Liu, Yue Ling, Yujia Qin, Zanbo Wang, Zhiwu He, Aoxue Zhang, Bairen Yi, Bencheng Liao, Can Huang, Can Zhang, Chaorui Deng, Chaoyi Deng, Cheng Lin, Cheng Yuan, Chenggang Li, Chenhui Gou, Chenwei Lou, Chengzhi Wei, Chundian Liu, Chunyuan Li, Deyao Zhu, Donghong Zhong, Feng Li, Feng Zhang, Gang Wu, Guodong Li, Guohong Xiao, Haibin Lin, Haihua Yang, Haoming Wang, Heng Ji, Hongxiang Hao, Hui Shen, Huixia Li, Jiahao Li, Jialong Wu, Jianhua Zhu, Jianpeng Jiao, Jiashi Feng, Jiaze Chen, Jianhui Duan, Jihao Liu, Jin Zeng, Jingqun Tang, Jingyu Sun, Joya Chen, Jun Long, Junda Feng, Junfeng Zhan, Junjie Fang, Junting Lu, Kai Hua, Kai Liu, Kai Shen, Kaiyuan Zhang, Ke Shen, Ke Wang, Keyu Pan, Kun Zhang, Kunchang Li, Lanxin Li, Lei Li, Lei Shi, Li Han, Liang Xiang, Liangqiang Chen, Lin Chen, Lin Li, Lin Yan, Liying Chi, Longxiang Liu, Mengfei Du, Mingxuan Wang, Ningxin Pan, Peibin Chen, Pengfei Chen, Pengfei Wu, Qingqing Yuan, Qingyao Shuai, Qiuyan Tao, Renjie Zheng, Renrui Zhang, Ru Zhang, Rui Wang, Rui Yang, Rui Zhao, Shaoqiang Xu, Shihao Liang, Shipeng Yan, Shu Zhong, Shuaishuai Cao, Shuangzhi Wu, Shufan Liu, Shuhan Chang, Songhua Cai, Tenglong Ao, Tianhao Yang, Tingting Zhang, Wanjun Zhong, Wei Jia, Wei Weng, Weihao Yu, Wenhao Huang, Wenjia Zhu, Wenli Yang, Wenzhi Wang, Xiang Long, XiangRui Yin, Xiao Li, Xiaolei Zhu, Xiaoying Jia, Xijin Zhang, Xin Liu, Xinchen Zhang, Xinyu Yang, Xiongcai Luo, Xiuli Chen, Xuantong Zhong, Xuefeng Xiao, Xujing Li, Yan Wu, Yawei Wen, Yifan Du, Yihao Zhang, Yining Ye, Yonghui Wu, Yu Liu, Yu Yue, Yufeng Zhou, Yufeng Yuan, Yuhang Xu, Yuhong Yang, Yun Zhang, Yunhao Fang, Yuntao Li, Yurui Ren, Yuwen Xiong, Zehua Hong, Zehua Wang, Zewei Sun, Zeyu Wang, Zhao Cai, Zhaoyue Zha, Zhecheng An, Zhehui Zhao, Zhengzhuo Xu, Zhipeng Chen, Zhiyong Wu, Zhuofan Zheng, Zihao Wang, Zilong Huang, Ziyu Zhu, Zuquan Song

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Large Language Model-Brained GUI Agents: A Survey

Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang

机构 * Microsoft（微软公司） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） ； Peking University（北京大学） 纠错

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments The collection of papers reviewed in this survey will be hosted and regularly updated on the GitHub repository: https://github.com/vyokky/LLM-Brained-GUI-Agents-Survey Additionally, a searchable webpage is available at https://aka.ms/gui-agent for easier access and exploration

MetaScenes: Towards Automated Replica Creation for Real-world 3D Scans

Huangyue Yu, Baoxiong Jia, Yixin Chen, Yandan Yang, Puhao Li, Rongpeng Su, Jiaxin Li, Qing Li, Wei Liang, Song-Chun Zhu, Tengyu Liu, Siyuan Huang

机构 * State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI） ； Beijing Institute of Technology（北京理工大学） ； Tsinghua University（清华大学） ； University of Science and Technology of China（中国科学技术大学） 纠错

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025

Constructive Approach to Bidirectional Influence between Qualia Structure and Language Emergence

Tadahiro Taniguchi, Masafumi Oizumi, Noburo Saji, Takato Horii, Naotsugu Tsuchiya

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks

Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U-Xuan Tan, Navonil Majumder, Soujanya Poria

机构 * Singapore University of Technology and Design（新加坡科技设计大学） ； Lambda Labs（Lambda实验室） 纠错

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs

Chun-Hsiao Yeh, Chenyu Wang, Shengbang Tong, Ta-Ying Cheng, Ruoyu Wang, Tianzhe Chu, Yuexiang Zhai, Yubei Chen, Shenghua Gao, Yi Ma

机构 * UC Berkeley（加州大学伯克利分校） ； TranscEngram ； NYU（纽约大学） ； University of Oxford（牛津大学） ； UC Davis（加州大学戴维斯分校） ； HKU（香港大学） 纠错

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Project page: https://danielchyeh.github.io/All-Angles-Bench/

AGI Is Coming... Right After AI Learns to Play Wordle

Sarath Shekkizhar, Romain Cosentino

机构 * Salesforce 纠错

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Understanding Epistemic Language with a Language-augmented Bayesian Theory of Mind

Lance Ying, Tan Zhi-Xuan, Lionel Wong, Vikash Mansinghka, Joshua B. Tenenbaum

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 23 pages; Published at the Transactions of the Association for Computational Linguistics (TACL); Presented at NAACL 2025

Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning

Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, Wenwu Zhu

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 5 figures

ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use

Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, Tat-Seng Chua

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments 13pages

Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactions

He Zhu, Quyu Kong, Kechun Xu, Xunlong Xia, Bing Deng, Jieping Ye, Rong Xiong, Yue Wang

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025

Unified Mind Model: Reimagining Autonomous Agents in the LLM Era

Pengbo Hu, Xiang Ying

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments 18 pages

Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis

Tingxuan Chen, Kun Yuan, Vinkle Srivastav, Nassir Navab, Nicolas Padoy

专题命中 多模态Agent ：image-text(abstract)；分类 cs.CV、cs.AI

SPA: 3D Spatial-Awareness Enables Effective Embodied Representation

Haoyi Zhu, Honghui Yang, Yating Wang, Jiange Yang, Limin Wang, Tong He

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://haoyizhu.github.io/spa/

MetaDesigner: Advancing Artistic Typography Through AI-Driven, User-Centric, and Multilingual WordArt Synthesis

Jun-Yan He, Zhi-Qi Cheng, Chenyang Li, Jingdong Sun, Qi He, Wangmeng Xiang, Hanyuan Chen, Jin-Peng Lan, Xianhui Lin, Kang Zhu, Bin Luo, Yifeng Geng, Xuansong Xie, Alexander G. Hauptmann

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.AI、cs.MM

Comments Accepted by ICLR 2025, Project: https://modelscope.cn/studios/WordArt/WordArt

Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokens

Ziwei Shan, Yaoyu He, Chengfeng Zhao, Jiashen Du, Jingyan Zhang, Qixuan Zhang, Jingyi Yu, Lan Xu

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments First three authors contribute equally. Project page: https://koyui.github.io/mojito/

The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning

Sheila Schoepp, Masoud Jafaripour, Yingyue Cao, Tianpei Yang, Fatemeh Abdollahi, Shadan Golestan, Zahin Sufiyan, Osmar R. Zaiane, Matthew E. Taylor

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 4 figures

A Roadmap for Embodied and Social Grounding in LLMs

Sara Incao, Carlo Mazzola, Giulia Belgiovine, Alessandra Sciutti

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted Version of a conference paper presented at Robophilosophy Conference 2024

Journal ref Incao, S., Mazzola, C., Belgiovine, G., Sciutti, A., 2025, A Roadmap for Embodied and Social Grounding in LLMs. In J. Seibt, P. Fazekas, & O. S. Quick (Eds.), Social Robots with AI: Prospects, Risks, and Responsible Methods, IOS Press

Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining

Zhiqi Ge, Juncheng Li, Xinglei Pang, Minghe Gao, Kaihang Pan, Wang Lin, Hao Fei, Wenqiao Zhang, Siliang Tang, Yueting Zhuang

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Developing Enhanced Conversational Agents for Social Virtual Worlds

D. Griol, A. Sanchis, J. M. Molina, Z. Callejas

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments Neurocomputing 2019

Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks

Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yan, Ji Zhang, Fei Huang, Heng Ji

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.CL

CrossVIT-augmented Geospatial-Intelligence Visualization System for Tracking Economic Development Dynamics

Yanbing Bai, Jinhua Su, Bin Qiao, Xiaoran Ma

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.AI

Mediating Modes of Thought: LLM's for design scripting

Moritz Rietschel, Fang Guo, Kyle Steinfeld

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Published at ACADIA 2024

BIP3D: Bridging 2D Images and 3D Perception for Embodied Intelligence

Xuewu Lin, Tianwei Lin, Lichao Huang, Hongyu Xie, Zhizhong Su

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CV、cs.AI

Neurosymbolic Graph Enrichment for Grounded World Models

Stefano De Giorgis, Aldo Gangemi, Alessandro Russo

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CL、cs.AI

A Review of Large Language Models and Autonomous Agents in Chemistry

Mayk Caldas Ramos, Christopher J. Collison, Andrew D. White

专题命中 多模态Agent ：multi-modal(abstract)；分类 cs.CL、cs.AI

To Ask or Not to Ask? Detecting Absence of Information in Vision and Language Navigation

Savitha Sam Abraham, Sourav Garg, Feras Dayoub

专题命中 多模态Agent ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at WACV 2025

Unlocking the Archives: Using Large Language Models to Transcribe Handwritten Historical Documents

Mark Humphries, Lianne C. Leddy, Quinn Downton, Meredith Legace, John McConnell, Isabella Murray, Elizabeth Spence

专题命中 多模态Agent ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 29 Pages, 11 Tables, 2 Figures