视觉大模型 / VLM

Can LLMs Generate Human-Like Wayfinding Instructions? Towards Platform-Agnostic Embodied Instruction Synthesis

Vishnu Sashank Dorbala, Sanjoy Chowdhury, Dinesh Manocha

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

Comments 14 Pages

Grounded Question-Answering in Long Egocentric Videos

Shangzhe Di, Weidi Xie

专题命中 视觉问答 ：grounding(abstract)；分类 cs.CV

Comments Accepted to CVPR 2024. Project website at https://dszdsz.cn/GroundVQA

Visual Superordinate Abstraction for Robust Concept Learning

Qi Zheng, Chaoyue Wang, Dadong Wang, Dacheng Tao

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 14 pages, 11 figures

Journal ref Machine Intelligence Research vol. 20, no. 1, pp. 79-91, 2024

Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA

Zhuowan Li, Bhavan Jasani, Peng Tang, Shabnam Ghadar

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted to CVPR 2024

CLAMP: Contrastive LAnguage Model Prompt-tuning

Piotr Teterwak, Ximeng Sun, Bryan A. Plummer, Kate Saenko, Ser-Nam Lim

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

ViT-Lens: Initiating Omni-Modal Exploration through 3D Insights

Weixian Lei, Yixiao Ge, Jianfeng Zhang, Dylan Sun, Kun Yi, Ying Shan, Mike Zheng Shou

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.CV

Comments 19 pages, 4 figures and 9 tables

Improved Baselines for Data-efficient Perceptual Augmentation of LLMs

Théophane Vallaeys, Mustafa Shukor, Matthieu Cord, Jakob Verbeek

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Few-Shot VQA with Frozen LLMs: A Tale of Two Approaches

Igor Sterner, Weizhe Lin, Jinghong Chen, Bill Byrne

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

GRAM: Global Reasoning for Multi-Page VQA

Tsachi Blau, Sharon Fogel, Roi Ronen, Alona Golts, Roy Ganz, Elad Ben Avraham, Aviad Aberdam, Shahar Tsiper, Ron Litman

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Knowledge Condensation and Reasoning for Knowledge-based VQA

Dongze Hao, Jian Jia, Longteng Guo, Qunbo Wang, Te Yang, Yan Li, Yanhua Cheng, Bo Wang, Quan Chen, Han Li, Jing Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Framework

Chris Kelly, Luhui Hu, Bang Yang, Yu Tian, Deshun Yang, Cindy Yang, Zaoshan Huang, Zihao Li, Jiayin Hu, Yuexian Zou

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 17 pages, 5 figures, and 1 table. arXiv admin note: substantial text overlap with arXiv:2311.10125

Beyond Text: Frozen Large Language Models in Visual Signal Comprehension

Lei Zhu, Fangyun Wei, Yanye Lu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

Answering Diverse Questions via Text Attached with Key Audio-Visual Clues

Qilang Ye, Zitong Yu, Xin Liu

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Training

Alyssa Huang, Peihan Liu, Ryumei Nakada, Linjun Zhang, Wanrong Zhang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Training Table Question Answering via SQL Query Decomposition

Raphaël Mouravieff, Benjamin Piwowarski, Sylvain Lamprier

专题命中 视觉问答 ：grounding(abstract)；分类 cs.AI

KVQ: Kwai Video Quality Assessment for Short-form Videos

Yiting Lu, Xin Li, Yajing Pei, Kun Yuan, Qizhi Xie, Yunpeng Qu, Ming Sun, Chao Zhou, Zhibo Chen

专题命中 视觉问答 ：vision language model(abstract)；分类 cs.CV

Comments 19 pages

Modularized Zero-shot VQA with Pre-trained Models

Rui Cao, Jing Jiang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments accepted as Findings in ACL 2023; Code available: https://github.com/abril4416/Mod-Zero-VQA

Question-Answer Cross Language Image Matching for Weakly Supervised Semantic Segmentation

Songhe Deng, Wei Zhuo, Jinheng Xie, Linlin Shen

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments ACM MM 2023

Gemini Pro Defeated by GPT-4V: Evidence from Education

Gyeong-Geon Lee, Ehsan Latif, Lehong Shi, Xiaoming Zhai

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI

NAAQA: A Neural Architecture for Acoustic Question Answering

Jerome Abdelnour, Jean Rouat, Giampiero Salvi

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.LG

Comments Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) in April 2021 (first revision February 2022)

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, Volume: 45 Issue: 4, Page(s): 4997-5009

Glance and Focus: Memory Prompting for Multi-Event Video Question Answering

Ziyi Bai, Ruiping Wang, Xilin Chen

专题命中 视觉问答 ：vision language model(abstract)；分类 cs.CV

Comments Accepted in NeurIPS 2023

Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Models

Xinpeng Ding, Jinahua Han, Hang Xu, Xiaodan Liang, Wei Zhang, Xiaomeng Li

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models

Zicheng Zhang, Haoning Wu, Zhongpeng Ji, Chunyi Li, Erli Zhang, Wei Sun, Xiaohong Liu, Xiongkuo Min, Fengyu Sun, Shangling Jui, Weisi Lin, Guangtao Zhai

专题命中 视觉问答 ：MLLM(abstract)；分类 cs.CV

OsmLocator: locating overlapping scatter marks with a non-training generative perspective

Yuming Qiu, Aleksandra Pizurica, Qi Ming, Nicolas Nadisic

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments 22pages

Reducing Hallucinations: Enhancing VQA for Flood Disaster Damage Assessment with Visual Contexts

Yimin Sun, Chao Wang, Yan Peng

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments already be accepted by 2024 3rd International Conference on Computer, Artificial Intelligence and Control Engineering (CAICE 2024)

Interactive Visual Task Learning for Robots

Weiwei Gu, Anant Sah, Nakul Gopalan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Comments In Proceedings of The 38th Annual AAAI Conference on Artificial Intelligence

VLIS: Unimodal Language Models Guide Multimodal Language Generation

Jiwan Chung, Youngjae Yu

专题命中 视觉问答 ：vision-language model(abstract)；分类 cs.AI

Comments Accepted as main paper in EMNLP 2023

VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generation

Jinguo Zhu, Xiaohan Ding, Yixiao Ge, Yuying Ge, Sijie Zhao, Hengshuang Zhao, Xiaohua Wang, Ying Shan

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.CV

Audio-Visual LLM for Video Understanding

Fangxun Shu, Lei Zhang, Hao Jiang, Cihang Xie

专题命中 视觉问答 ：multimodal large language model(abstract)；分类 cs.CV

Causal-CoG: A Causal-Effect Look at Context Generation for Boosting Multi-modal Language Models

Shitian Zhao, Zhuowan Li, Yadong Lu, Alan Yuille, Yan Wang

专题命中 视觉问答 ：visual question answering(abstract)；分类 cs.AI