视觉大模型 / VLM

Towards Reliable and Interpretable Document Question Answering via VLMs

Alessio Chen, Simone Giovannini, Andrea Gemelli, Fabio Coppini, Simone Marinai

机构 * Università degli Studi di Firenze（佛罗伦萨大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；grounding(abstract)

FM2DS: Few-Shot Multimodal Multihop Data Synthesis with Knowledge Distillation for Question Answering

Amirhossein Abaskohi, Spandana Gella, Giuseppe Carenini, Issam H. Laradji

机构 * Department of Computer Science（计算机科学系） ； The University of British Columbia（不列颠哥伦比亚大学） ； ServiceNow Research（ServiceNow研究） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Findings of EMNLP 2025

Real Time FPGA Based Transformers & VLMs for Vision Tasks: SOTA Designs and Optimizations

Safa Mohammed Sali, Mahmoud Meribout, Ashiyana Abdul Majeed

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)

Towards Understanding Camera Motions in Any Video

Zhiqiu Lin, Siyuan Cen, Daniel Jiang, Jay Karhade, Hewei Wang, Chancharik Mitra, Tiffany Ling, Yuhan Huang, Sifan Liu, Mingyu Chen, Rushikesh Zawar, Xue Bai, Yilun Du, Chuang Gan, Deva Ramanan

专题命中 视觉问答 ：VLM(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Project site: https://linzhiqiu.github.io/papers/camerabench/

Do LVLMs Know What They Know? A Systematic Study of Knowledge Boundary Perception in LVLMs

Zhikai Ding, Shiyu Ni, Keping Bi

机构 * State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院人工智能安全重点实验室） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；visual question answering(abstract)

Comments EMNLP2025 Findings

Hierarchical Vision-Language Reasoning for Multimodal Multiple-Choice Question Answering

Ao Zhou, Zebo Gu, Tenghao Sun, Jiawen Chen, Mingsheng Tu, Zifeng Cheng, Yafeng Yin, Zhiwei Jiang, Qing Gu

机构 * State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学） ； Chongqing University of Posts and Telecommunications（重庆邮电大学） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；multimodal large language model(abstract)

Comments This paper has been accepted by ACM MM 2025

SVLA: A Unified Speech-Vision-Language Assistant with Multimodal Reasoning and Speech Generation

Ngoc Dung Huynh, Mohamed Reda Bouadjenek, Imran Razzak, Hakim Hacid, Sunil Aryal

专题命中 视觉问答 ：vision language model(abstract)；visual question answering(abstract)

Comments 21 pages

Directional Gradient Projection for Robust Fine-Tuning of Foundation Models

Chengyue Huang, Junjiao Tian, Brisa Maneechotesuwan, Shivang Chopra, Zsolt Kira

机构 * Georgia Institute of Technology（佐治亚理工学院） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to ICLR 2025

SlotPi: Physics-informed Object-centric Reasoning Models

Jian Li, Wan Han, Ning Lin, Yu-Liang Zhan, Ruizhi Chengze, Haining Wang, Yi Zhang, Hongsheng Liu, Zidong Wang, Fan Yu, Hao Sun

机构 * Renmin University of China（中国人民大学） ； Huawei Technologies Ltd.（华为技术有限公司） 纠错

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering

Jingxuan Wei, Nan Xu, Junnan Zhu, Yanni Hao, Gaowei Wu, Bihui Yu, Lei Wang

机构 * Beijing Wenge Technology Co., Ltd.（北京文格科技有限公司） ； Shenyang Institute of Computing Technology, Chinese Academy of Sciences（沈阳计算技术研究所，中国科学院） ； MAIS, Institute of Automation, Chinese Academy of Sciences（自动化研究所，中国科学院） ； University of Chinese Academy of Sciences（中国科学院大学） 纠错

专题命中 视觉问答 ：vision-language model(abstract)；multimodal large language model(abstract)

FullFront: Benchmarking MLLMs Across the Full Front-End Engineering Workflow

Haoyu Sun, Huichen Will Wang, Jiawei Gu, Linjie Li, Yu Cheng

机构 * Tongji University（同济大学） ； University of Washington（华盛顿大学） ； Sun Yat-sen University（中山大学） ； Microsoft（微软） ； The Chinese University of Hong Kong（香港中文大学） 纠错

专题命中 视觉问答 ：multimodal large language model(abstract)；MLLM(abstract)

CTRL-O: Language-Controllable Object-Centric Visual Representation Learning

Aniket Didolkar, Andrii Zadaianchuk, Rabiul Awal, Maximilian Seitzer, Efstratios Gavves, Aishwarya Agrawal

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted at CVPR 2025

MoReVQA: Exploring Modular Reasoning Models for Video Question Answering

Juhong Min, Shyamal Buch, Arsha Nagrani, Minsu Cho, Cordelia Schmid

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR 2024; updated NExT-GQA results in Appendix

PAVE: Patching and Adapting Video Large Language Models

Zhuoming Liu, Yiquan Li, Khoi Duc Nguyen, Yiwu Zhong, Yin Li

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR2025 Camera Ready

DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models

Yongdong Wang, Runze Xiao, Jun Younes Louhi Kasahara, Ryosuke Yajima, Keiji Nagatani, Atsushi Yamashita, Hajime Asama

专题命中 视觉问答 ：vision-language model(abstract)；VLM(abstract)

Comments The work was first submitted to an IEEE conference on September 15, 2024

Adaptive Keyframe Sampling for Long Video Understanding

Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, Qixiang Ye

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments CVPR2025

Bayesian Low-Rank LeArning (Bella): A Practical Approach to Bayesian Neural Networks

Bao Gia Doan, Afshar Shamsi, Xiao-Yu Guo, Arash Mohammadi, Hamid Alinejad-Rokny, Dino Sejdinovic, Damien Teney, Damith C. Ranasinghe, Ehsan Abbasnejad

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This paper is accepted in AAAI'25", and the code is available at https://bnn-bella.github.io/BNN-Bella/

Reasoning-Enhanced Object-Centric Learning for Videos

Jian Li, Pu Ren, Yang Liu, Hao Sun

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimization

Yue Zhang, Liqiang Jing, Vibhav Gogate

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted by AAAI 2025

VQA-Levels: A Hierarchical Approach for Classifying Questions in VQA

Madhuri Latha Madaka, Chakravarthy Bhagvati

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data

Jeremy Andrew Irvin, Emily Ruoyu Liu, Joyce Chuyi Chen, Ines Dormoy, Jinyoung Kim, Samar Khanna, Zhuo Zheng, Stefano Ermon

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published at ICLR 2025

Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy

Priyaranjan Pattnayak, Hitesh Laxmichand Patel, Bhargava Kumar, Amit Agarwal, Ishan Banerjee, Srikant Panda, Tejaswini Kumar

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

NoisyEQA: Benchmarking Embodied Question Answering Against Noisy Queries

Tao Wu, Chuhao Zhou, Yen Heng Wong, Lin Gu, Jianfei Yang

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV、cs.AI、cs.LG

Uni-Mlip: Unified Self-supervision for Medical Vision Language Pre-training

Ameera Bawazir, Kebin Wu, Wenbin Li

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments 15 pages, 2 figures, accepted by BMVC'24

Preserving Pre-trained Representation Space: On Effectiveness of Prefix-tuning for Large Multi-modal Models

Donghoon Kim, Gusang Lee, Kyuhong Shim, Byonghyo Shim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Findings of EMNLP 2024

Visual Text Matters: Improving Text-KVQA with Visual Text Entity Knowledge-aware Large Multimodal Assistant

Abhirama Subramanyam Penamakuri, Anand Mishra

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Accepted to EMNLP (Main) 2024

Sparks of Artificial General Intelligence(AGI) in Semiconductor Material Science: Early Explorations into the Next Frontier of Generative AI-Assisted Electron Micrograph Analysis

Sakhinana Sagar Srinivas, Geethan Sannidhi, Sreeja Gangasani, Chidaksh Ravuru, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Published at Deployable AI (DAI) Workshop at AAAI-2024

A Multimodal Memes Classification: A Survey and Open Research Issues

Tariq Habib Afridi, Aftab Alam, Muhammad Numan Khan, Jawad Khan, Young-Koo Lee

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments This is a survey paper on recent state of the art VL models that can be used for memes classification. it has 15 pages and 2 figures

Journal ref SCA 2020

Towards Human-Level Understanding of Complex Process Engineering Schematics: A Pedagogical, Introspective Multi-Agent Framework for Open-Domain Question Answering

Sagar Srinivas Sakhinana, Geethan Sannidhi, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Our paper is accepted for publication at ML4CCE workshop at ECML PKDD 2024

Foundational Model for Electron Micrograph Analysis: Instruction-Tuning Small-Scale Language-and-Vision Assistant for Enterprise Adoption

Sakhinana Sagar Srinivas, Chidaksh Ravuru, Geethan Sannidhi, Venkataramana Runkana

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV、cs.AI、cs.LG

Comments Our paper is published at ICML 2024 Workshop ML for Life and Material Science: From Theory to Industry Applications, Vienna, Austria