ContextNav: Towards Agentic Multimodal In-Context Learning
The University of Queensland(昆士兰大学);Nanjing University(南京大学);University of California, Los Angeles(加州大学洛杉矶分校);University of California, Merced(加州大学默塞德分校)
Honghao Fu, Yuan Ouyang, Kai-Wei Chang, Yiwei Wang, Zi Huang, Yujun Cai
Recent advances demonstrate that multimodal large language models (MLLMs) exhibit strong multimodal in-context learning (ICL) capabilities, enabling them to adapt to novel vision-language tasks from a few contextual examples. However, existing ICL approaches face challenges in reconciling scalability with robustness across diverse tasks and noisy contextual examples: manually selecting examples produces clean contexts but is labor-intensive and task-specific, while similarity-based retrieval improves scalability but could introduce irrelevant or structurally inconsistent samples that degrade ICL performance. To address these limitations, we propose ContextNav, the first agentic framework that integrates the scalability of automated retrieval with the quality and adaptiveness of human-like curation, enabling noise-robust and dynamically optimized contextualization for multimodal ICL. ContextNav unifies context management and noise-robust contextualization within a closed-loop workflow driven by graph-based orchestration. Specifically, it builds a resource-aware multimodal embedding pipeline, maintains a retrievable vector database, and applies agentic retrieval and structural alignment to construct noise-resilient contexts. An Operational Grammar Graph (OGG) further supports adaptive workflow planning and optimization, enabling the agent to refine its operational strategies based on downstream ICL feedback. Experimental results demonstrate that ContextNav achieves state-of-the-art performance across various datasets, underscoring the promise of agentic workflows for advancing scalable and robust contextualization in multimodal ICL.
Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Framework
University College London(伦敦大学学院);National Health Service(英国国家医疗服务体系);University of Surrey(萨里大学);Samsung AI Centre(三星人工智能中心);Queen Mary University of London(伦敦玛丽女王大学)
MARCUS: An agentic, multimodal vision-language model for cardiac diagnosis and management
Division of Cardiology, Department of Medicine, Stanford University(斯坦福大学心脏病学系);Department of Biomedical Data Science, Stanford University(斯坦福大学生物医学数据科学系);Department of Medicine, Radiology, and Pediatrics, UCSF(旧金山大学医学系、放射学与儿科学系);Bakar Institute, UCSF(Bakar研究所,旧金山大学);UCSF–UC Berkeley Joint Program in Computational Precision Health(旧金山大学-伯克利计算精准健康联合计划);Department of Radiology, Stanford University(斯坦福大学放射学系);Department of Psychiatry and Behavioral Sciences, Stanford University(斯坦福大学精神病学与行为科学系);Department of Computer Science, Stanford University(斯坦福大学计算机科学系);Department of Electrical Engineering, Stanford University(斯坦福大学电气工程系);Department of Biology, Stanford University(斯坦福大学生物学系)
Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning
School of Artificial Intelligence UCAS(中国科学院大学人工智能学院);Institute of Automation CAS(中国科学院自动化研究所);Ant Digital Technologies Ant Group(蚂蚁数字科技蚂蚁集团);RUC(中国人民大学);BIT(北京理工大学)
Towards Agentic Recommender Systems in the Era of Multimodal Large Language Models
University of New South Wales(新南威尔士大学);University of California San Diego(加利福尼亚大学圣迭戈分校);Indiana University(印第安纳大学);Adobe Research(奥多比研究院);CSIRO’s Data61(澳大利亚联邦科学与工业研究组织Data61研究院)
A Survey on Agentic Multimodal Large Language Models
Nanyang Technological University, Singapore(南洋理工大学,新加坡);Chinese University of Hong Kong, Shenzhen, China(香港中文大学(深圳));Shenzhen Research Institute of Big Data, China(大数据研究 institute);Sun Yat-sen University, China(中山大学);City University of Hong Kong, China(香港城市大学);Communication University of China, China(通信大学)
StarDojo: Benchmarking Open-Ended Behaviors of Agentic Multimodal LLMs in Production-Living Simulations with Stardew Valley
Nanyang Technological University(南洋理工大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));East China University of Science and Technology(华东理工大学);Singapore Management University(新加坡管理大学)
SurgVisAgent: Multimodal Agentic Model for Versatile Surgical Visual Enhancement
University of Chinese Academy of Sciences Multimedia Department, Xiaomi Inc Chinese Academy of Sciences, Institute of Automation Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences
Volumetric Radiology AI in the Era of Multimodal Large Language Models
Fudan University(复旦大学);Northwestern Polytechnical University(西北工业大学);Xi’an Jiaotong University(西安交通大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Southern Medical University(南方医科大学);The Chinese University of Hong Kong(香港中文大学);University of British Columbia(不列颠哥伦比亚大学);Microsoft Research(微软研究院);Westlake University(西湖大学)
PathoSage: Towards Multi-Source Evidence Adjudication in Pathology via Experience-Aware Agentic Workflow
College of Computer Science, Sichuan University(四川大学计算机科学学院);Department of Pathology and Institute of Clinical Pathology, West China Hospital, Sichuan University(四川大学华西医院病理科/临床病理研究所);Department of Computer and Information Science, University of Macau(澳门大学计算机与信息科学系);School of Intelligent Systems Engineering, Sun Yat-sen University(中山大学智能工程学院)
Octopus: Agentic Multimodal Reasoning with Six-Capability Orchestration
Sun Yat-sen University(中山大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University(浙江大学);Tsinghua University(清华大学);Peking University(北京大学);The Hong Kong University of Science and Technology(香港科技大学);University of Chinese Academy of Sciences(中国科学院大学)
See, Hypothesize, Validate: Multimodal Agentic Framework for Discovering Governing PDEs
Indian Institute of Technology Delhi(印度德里理工学院);Robert Bosch GmbH(罗伯特·博世有限公司);TCS Research(塔塔咨询服务公司研究院);Yardi School of Artificial Intelligence (ScAI)(亚迪人工智能学院);Department of Applied Mechanics(应用力学系)
Agentic Learner with Grow-and-Refine Multimodal Semantic Memory
Nanjing University of Science and Technology(南京理工大学);Baidu Inc(百度公司);AIML, Adelaide University(AIML,阿德莱德大学);Singapore University of Technology and Design(新加坡科技设计大学)
MANA: Towards Efficient Mobile Ad Detection via Multimodal Agentic UI Navigation
Department of Computer Science and Technology(计算机科学与技术系);University of Southern California(南加州大学);Shanghai Jiao Tong University(上海交通大学);State Key Laboratory of Internet Architecture(互联网架构国家重点实验室)
FactGuard: Agentic Video Misinformation Detection via Reinforcement Learning
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China(中国科学院计算技术研究所);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学);University of Science and Technology Beijing(北京科技大学);The University of Queensland, Brisbane, Australia(昆士兰大学)
Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generation
National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);University of Edinburgh(爱丁堡大学);Beijing Institute of Technology(北京理工大学)