Visual Large Language Models for Generalized and Specialized Applications
Michigan State University（密歇根州立大学）；Cornell University（康奈尔大学）；Arizona State University（亚利桑那州立大学）；University of California, Berkeley（加州大学伯克利分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Yale University（耶鲁大学）
Yifan Li, Zhixin Lai, Wentao Bao, Zhen Tan, Anh Dao, Kewei Sui, Jiayi Shen, Dong Liu, Huan Liu, Yu Kong
Visual-language models (VLM) have emerged as a powerful tool for learning a unified embedding space for vision and language. Inspired by large language models, which have demonstrated strong reasoning and multi-task capabilities, visual large language models (VLLMs) are gaining increasing attention for building general-purpose VLMs. Despite the significant progress made in VLLMs, the related literature remains limited, particularly from a comprehensive application perspective, encompassing generalized and specialized applications across vision (image, video, depth), action, and language modalities. In this survey, we focus on the diverse applications of VLLMs, examining their using scenarios, identifying ethics consideration and challenges, and discussing future directions for their development. By synthesizing these contents, we aim to provide a comprehensive guide that will pave the way for future innovations and broader applications of VLLMs. The paper list repository is available: https://github.com/JackYFL/awesome-VLLMs.
ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering
Digital Research Center of Sfax, SMARTS Laboratory（突尼斯斯法克斯数字研究中心，SMARTS实验室）；École Polytechnique de Tunisie, University of Carthage（突尼斯理工学院，卡塔赫纳大学）；Computer Vision Center, Universitat Autònoma de Barcelona（巴塞罗那自治大学计算机视觉中心）
Boosting Visual Knowledge-Intensive Training for LVLMs Through Causality-Driven Visual Object Completion
School of Informatics, Xiamen University, China（厦门大学信息学院）；Xiamen Unisound Intelligence Technology Co., Ltd（厦门Unisound智能科技有限公司）；Shanghai Artificial Intelligence Laboratory, China（上海人工智能实验室）；Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China（福建省和台湾非物质文化遗产数字化保护与智能处理重点实验室（厦门大学），中华人民共和国文化和旅游部，中国）
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding
Nanyang Technological University（南洋理工大学）；University of Reading（阅读大学）；Shanghai Jiao Tong University（上海交通大学）；Harbin Institute of Technology（哈尔滨工业大学）；SenseTime Research（商汤科技研究院）
Generalized Hadamard-Product Fusion Operators for Visual Question Answering
School of Engineering, University of Guelph（圭尔夫大学工程学院）；Vector Institute for Artificial Intelligence（向量人工智能研究所）；Canadian Institute for Advanced Research（加拿大高级研究学院(注：原机构名常译为加拿大高级研究所，此处按原文准确翻译)）
Picking the Right Specialist: Attentive Neural Process-based Selection of Task-Specialized Models as Tools for Agentic Healthcare Systems
Department of Engineering Science, University of Oxford, United Kingdom（牛津大学工程科学系）；Department of Computer Science, Khalifa University, Abu Dhabi, United Arab Emirates（哈利法大学计算机科学系）
Distilling Specialized Orders for Visual Generation
Stony Brook University（石溪大学）；International Laboratory on Learning Systems (ILLS)（国际学习系统实验室（ILLS））；LIVIA, ÉTS Montréal（LIVIA，蒙特利尔高等技术学院）；Mila–Quebec AI Institute（米拉-魁北克人工智能研究所）；ServiceNow Research（ServiceNow研究院）；Université Paris-Saclay, CentraleSupélec（巴黎-萨克雷大学，中央理工-高等电力学院）；Polytechnique Montréal（蒙特利尔综合理工学院）；Canada CIFAR AI Chair（加拿大CIFAR人工智能主席）
Unifying Specialized Visual Encoders for Video Language Models
Department of Computer Science, Princeton University, Princeton, NJ, United States（普林斯顿大学计算机科学系）；Salesforce Research, Palo Alto, CA, United States（Salesforce研究）
AnomalyMoE: Towards a Language-free Generalist Model for Unified Visual Anomaly Detection
Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所基础模型研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China（中国科学院大学人工智能学院）；Objecteye Inc., Beijing, China（Objecteye公司）
Xi’an Jiaotong University（西安交通大学）；Tsinghua University（清华大学）；DAMO Academy, Alibaba Group（阿里云实验室）；Chinese Academy of Sciences（中国科学院）；National University of Singapore（新加坡国立大学）
Domain-aware Visual Bias Eliminating for Generalized Zero-Shot Learning
University of Science and Technology of China（中国科学技术大学）；National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）
