Training vision-language models via instruction tuning relies on large data mixtures spanning diverse tasks and domains, yet these mixtures frequently include redundant information that increases computational costs without proportional gains. Existing methods typically rely on task-agnostic heuristics to estimate data importance, limiting their effectiveness across tasks. We introduce ICONS, a gradient-based Influence CONsensus approach for vision-language data Selection. Our method leverages first-order training dynamics to estimate each example's influence on validation performance, then aggregates these estimates across tasks via majority voting. This cross-task consensus identifies consistently valuable data points while mitigating score calibration and outlier sensitivity, enabling robust and scalable data selection for diverse multitask mixtures. Models trained on our selected 20% data subset from LLAVA-665K (respectively: from CAMBRIAN-7M, from VISION-FLAN-186K) retain 98.6% (respectively: 98.8%, 99.8%) of full-dataset performance. We demonstrate that our selected data generalizes to unseen tasks and model architectures, and release three compact subsets LLAVA-ICONS-133K, CAMBRIAN-ICONS-1.4M, and VISION-FLAN-ICONS-37K for efficient vision-language model development.
The Visual Iconicity Challenge: Evaluating Vision-Language Models on Sign Language Form-Meaning Mapping
Multimodal Language Department(多模态语言部门);Max Planck Institute for Psycholinguistics(马克斯·普朗克心理语言学研究所);Department of Linguistics(语言学系);Boğaziçi University(博多伊奇大学);Donders Institute for Brain Cognition and Behaviour(多纳尔斯脑认知与行为研究所);Radboud University(拉德堡德大学);Department of Linguistics and Communication(语言学与沟通系);University of Birmingham(伯明翰大学)
Vision Language Models Map Logos to Text via Semantic Entanglement in the Visual Projector
University of California, Merced(加州大学梅尔德分校);vivo Mobile Communication Co., Ltd.(vivo移动通信有限公司);University of Queensland(昆士兰大学);UCLA(加州大学洛杉矶分校);University at Buffalo(布法罗大学)
VLM-CPL: Consensus Pseudo Labels from Vision-Language Models for Annotation-Free Pathological Image Classification
School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China(电子科技大学机械与电子工程学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Department of Pathology, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Affiliated Cancer Hospital of University of Electronic Science and Technology of China(pathology department, 四川省癌症临床研究中心, 四川省肿瘤医院及研究所, 电子科技大学附属肿瘤医院);Department of Radiation Oncology, Sichuan Cancer Hospital and Institute, University of Electronic Science and Technology of China(放射肿瘤科, 四川省肿瘤医院及研究所, 电子科技大学)
Benchmarking Vision-Language and Multimodal Large Language Models in Zero-shot and Few-shot Scenarios: A study on Christian Iconography
Gianmarco Spinaci Department of Classical Philology and Italian Studies, University of Bologna, Italy Villa i Tatti, The Harvard University Center for Italian Renaissance Studies, Florence, Italy(Gianmarco Spinaci 文艺复兴研究系,博洛尼亚大学,意大利 塔蒂别墅,哈佛大学意大利文艺复兴研究中心,佛罗伦萨,意大利);Lukas Klic Villa i Tatti, The Harvard University Center for Italian Renaissance Studies, Florence, Italy(Lukas Klic 塔蒂别墅,哈佛大学意大利文艺复兴研究中心,佛罗伦萨,意大利);Giovanni Colavizza Department of Classical Philology and Italian Studies, University of Bologna, Italy Department of Communication, University of Copenhagen, Denmark(Giovanni Colavizza 文艺复兴研究系,博洛尼亚大学,意大利 传播系,哥本哈根大学,丹麦)
Faculty of Computer Science, University of Vienna(维也纳大学计算机科学学院);UniVie Doctoral School Computer Science, University of Vienna(维也纳大学UniVie计算机科学博士学院);LMU Munich(慕尼黑大学);Munich Center for Machine Learning(慕尼黑机器学习中心);Faculty of Philological and Cultural Studies, University of Vienna(维也纳大学语言与文化学院)
The Influence of Iconicity in Transfer Learning for Sign Language Recognition
ADAPT Centre, Munster Technological University, Cork, Ireland(ADAPT中心,穆斯廷技术大学,科克,爱尔兰);Nimbus Centre, Munster Technological University, Cork, Ireland(Nimbus中心,穆斯廷技术大学,科克,爱尔兰);School of Electronics, Electrical Engineering and Computer Science (EEECS), Queen's University Belfast, UK(电子、电气与计算机科学学院(EEECS),女王大学贝尔法斯特,英国)
CARE: Class-Adaptive Expert Consensus for Reliable Learning with Long-Tailed Noisy Labels
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China.(深圳大学计算机科学与软件工程学院,中国深圳);Guangming Laboratory, Shenzhen, China.(深圳广明实验室,中国深圳);School of Informatics, Xiamen University, Xiamen, China.(厦门大学信息学院,中国厦门);School of Computer Science and Technology, Guangdong University of Technology, Guangzhou, China(广东工业大学计算机科学与技术学院,中国广州)
Closed-Loop Bidirectional Prompting for Adversarial Robustness of Vision Language Models
University of Macau(澳门大学);Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Peking University(北京大学);Independent Researcher(独立研究员);Institute of Science Tokyo(东京科学研究院);Morgan Stanley(摩根大通);Halmstad University(哈马碧大学)
Trifuse: Enhancing Attention-Based GUI Grounding via Multimodal Fusion
College of Computer Science and Technology, National University of Defense Technology(计算机科学与技术学院,国防科技大学);Intelligent Game and Decision Lab, Academy of Military Sciences(智能游戏与决策实验室,军事科学院)
Department of Computer Science, University of British Columbia(不列颠哥伦比亚大学计算机科学系);Vector Institute for AI(人工智能矢量研究所);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)