视觉大模型 / VLM

Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery

Long Bai, Guankun Wang, Mobarakol Islam, Lalithkumar Seenivasan, An Wang, Hongliang Ren

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by Information Fusion. Code and data availability: https://github.com/longbai1006/Surgical-VQLAPlus

Look, Learn and Leverage (L$^3$): Mitigating Visual-Domain Shift and Discovering Intrinsic Relations via Symbolic Alignment

Hanchen Xie, Jiageng Zhu, Mahyar Khayatkhoei, Jiazhi Li, Wael AbdAlmageed

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 17 pages, 9 figures, 6 tables

Can SAR improve RSVQA performance?

Lucrezia Tosato, Sylvain Lobry, Flora Weissgerber, Laurent Wendling

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 6 pages, 4 figures

Journal ref 15th European Conference on Synthetic Aperture Radar, April 23 26, 2024, Munich, Germany

Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos

Qirui Chen, Shangzhe Di, Weidi Xie

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models

Yangyang Guo, Fangkai Jiao, Zhiqi Shen, Liqiang Nie, Mohan Kankanhalli

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by TPAMI

KOSMOS-2.5: A Multimodal Literate Model

Tengchao Lv, Yupan Huang, Jingye Chen, Yuzhong Zhao, Yilin Jia, Lei Cui, Shuming Ma, Yaoyao Chang, Shaohan Huang, Wenhui Wang, Li Dong, Weiyao Luo, Shaoxiang Wu, Guoxin Wang, Cha Zhang, Furu Wei

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

FEDMEKI: A Benchmark for Scaling Medical Foundation Models via Federated Knowledge Injection

Jiaqi Wang, Xiaochen Wang, Lingjuan Lyu, Jinghui Chen, Fenglong Ma

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments Submitted to Neurips 2024 DB Track

Q-Bench+: A Benchmark for Multi-modal Foundation Models on Low-level Vision from Single Images to Pairs

Zicheng Zhang, Haoning Wu, Erli Zhang, Guangtao Zhai, Weisi Lin

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by TPAMI. arXiv admin note: substantial text overlap with arXiv:2309.14181

BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks

Kai Zhang, Rong Zhou, Eashan Adhikarla, Zhiling Yan, Yixin Liu, Jun Yu, Zhengliang Liu, Xun Chen, Brian D. Davison, Hui Ren, Jing Huang, Chen Chen, Yuyin Zhou, Sunyang Fu, Wei Liu, Tianming Liu, Xiang Li, Yong Chen, Lifang He, James Zou, Quanzheng Li, Hongfang Liu, Lichao Sun

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments Fix incorrect citations and add journal reference for the published version. Nat Med (2024)

Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations

Yongshuo Zong, Tingyang Yu, Ruchika Chavhan, Bingchen Zhao, Timothy Hospedales

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.LG

Comments ICML 2024; v3 fix typo

Question-Instructed Visual Descriptions for Zero-Shot Video Question Answering

David Romero, Thamar Solorio

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

QuIIL at T3 challenge: Towards Automation in Life-Saving Intervention Procedures from First-Person View

Trinh T. L. Vuong, Doanh C. Bui, Jin Tae Kwak

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments MICCAI-Thompson Challenge 2023

Diffusion-Refined VQA Annotations for Semi-Supervised Gaze Following

Qiaomu Miao, Alexandros Graikos, Jingwei Zhang, Sounak Mondal, Minh Hoai, Dimitris Samaras

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to ECCV 2024

VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding

Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, Qing Li

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments ECCV-24; Project page: videoagent.github.io; First two authors contributed equally

VQA-Diff: Exploiting VQA and Diffusion for Zero-Shot Image-to-3D Vehicle Asset Generation in Autonomous Driving

Yibo Liu, Zheyuan Yang, Guile Wu, Yuan Ren, Kejian Lin, Bingbing Liu, Yang Liu, Jinjun Shan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Recursive Visual Programming

Jiaxin Ge, Sanjay Subramanian, Baifeng Shi, Roei Herzig, Trevor Darrell

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Overview of BioASQ 2022: The tenth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

Anastasios Nentidis, Georgios Katsimpras, Eirini Vandorou, Anastasia Krithara, Antonio Miranda-Escalada, Luis Gasco, Martin Krallinger, Georgios Paliouras

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

Comments 25 pages, 14 tables, 4 figures. arXiv admin note: substantial text overlap with arXiv:2106.14885

Journal ref Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2022. Lecture Notes in Computer Science, vol 13390. Springer, Cham

SADL: An Effective In-Context Learning Method for Compositional Visual QA

Long Hoang Dang, Thao Minh Le, Vuong Le, Tu Minh Phuong, Truyen Tran

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

BioKGBench: A Knowledge Graph Checking Benchmark of AI Agent for Biomedical Science

Xinna Lin, Siqi Ma, Junjie Shan, Xiaojing Zhang, Shell Xu Hu, Tiannan Guo, Stan Z. Li, Kaicheng Yu

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

Assistive Image Annotation Systems with Deep Learning and Natural Language Capabilities: A Review

Moseli Mots'oehli

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted IEEE ETNCC 2024, 9 pages

Selectively Answering Visual Questions

Julian Martin Eisenschlos, Hernán Maina, Guido Ivetta, Luciana Benotti

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments To be published in the findings of the 2024 Annual Meeting of the Association for Computational Linguistics

Video Question Answering for People with Visual Impairments Using an Egocentric 360-Degree Camera

Inpyo Song, Minjun Joo, Joonhyung Kwon, Jangwon Lee

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR2024 EgoVis Workshop

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark

Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, Limin Wang, Yu Qiao

专题命中 视觉问答 ：MLLM(abstract)；分类 cs.CV

Comments CVPR 2024 highlight: updated version with Mistral and better performances for MVBench/NExT-QA/STAR/TVQA/EgoSchema/IntentQA

Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding

Zheng Zhao, Emilio Monti, Jens Lehmann, Haytham Assem

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

Comments Accepted to NAACL 2024

MovieChat+: Question-aware Sparse Memory for Long Video Question Answering

Enxin Song, Wenhao Chai, Tian Ye, Jenq-Neng Hwang, Xi Li, Gaoang Wang

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

MM-PhyQA: Multimodal Physics Question-Answering With Multi-Image CoT Prompting

Avinash Anand, Janak Kapuriya, Apoorv Singh, Jay Saraf, Naman Lal, Astha Verma, Rushali Gupta, Rajiv Shah

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.AI

CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update

Zhi Gao, Yuntao Du, Xintong Zhang, Xiaojian Ma, Wenjuan Han, Song-Chun Zhu, Qing Li

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2024

Mitigating the Impact of Attribute Editing on Face Recognition

Sudipta Banerjee, Sai Pranaswi Mullangi, Shruti Wagle, Chinmay Hegde, Nasir Memon

专题命中 视觉问答 ：LLaVA(abstract)；分类 cs.CV

Comments Under review

Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learning

Juncheng Yang, Zuchao Li, Shuai Xie, Wei Yu, Shijun Li, Bo Du

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments This paper is accepted to LREC-COLING 2024

Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement

Zaid Khan, Vijay Kumar BG, Samuel Schulter, Yun Fu, Manmohan Chandraker

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments CVPR 2024