视觉大模型 / VLM

The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks

Isaac R. Galatzer-Levy, David Munday, Jed McGiffin, Xin Liu, Danny Karmon, Ilia Labzovsky, Rivka Moroshko, Amir Zait, Daniel McDuff

专题命中 视觉推理 ：vision language model(abstract)；分类 cs.AI

ReFeR: Improving Evaluation and Reasoning through Hierarchy of Models

Yaswanth Narsupalli, Abhranil Chandra, Sreevatsa Muppirala, Manish Gupta, Pawan Goyal

专题命中 视觉推理 ：vision language model(abstract)；分类 cs.AI

Comments Paper Under Review

Temporal Reasoning Transfer from Text to Video

Lei Li, Yuanxin Liu, Linli Yao, Peiyuan Zhang, Chenxin An, Lean Wang, Xu Sun, Lingpeng Kong, Qi Liu

专题命中 视觉推理 ：InternVL(abstract)；分类 cs.CV

Comments Project page: https://video-t3.github.io

On Efficient Language and Vision Assistants for Visually-Situated Natural Language Understanding: What Matters in Reading and Reasoning

Geewook Kim, Minjoon Seo

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments EMNLP 2024 Main

Reasoning with Natural Language Explanations

Marco Valentino, André Freitas

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI

Comments Tutorial to be presented at EMNLP 2024. Website: https://sites.google.com/view/reasoning-with-explanations

VDebugger: Harnessing Execution Feedback for Debugging Visual Programs

Xueqing Wu, Zongyu Lin, Songyan Zhao, Te-Lin Wu, Pan Lu, Nanyun Peng, Kai-Wei Chang

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV

Comments EMNLP 2024 Findings

ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigation

Abrar Anwar, John Welsh, Joydeep Biswas, Soha Pouya, Yan Chang

专题命中 视觉推理 ：VLM(abstract)；分类 cs.AI

Impact of ML Optimization Tactics on Greener Pre-Trained ML Models

Alexandra González Álvarez, Joel Castaño, Xavier Franch, Silverio Martínez-Fernández

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.LG

MAGDA: Multi-agent guideline-driven diagnostic assistance

David Bani-Harouni, Nassir Navab, Matthias Keicher

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.AI

PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis

Meng Luo, Hao Fei, Bobo Li, Shengqiong Wu, Qian Liu, Soujanya Poria, Erik Cambria, Mong-Li Lee, Wynne Hsu

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.AI

Comments Accepted by ACM MM 2024 (Oral)

X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoning

Artemis Panagopoulou, Le Xue, Ning Yu, Junnan Li, Dongxu Li, Shafiq Joty, Ran Xu, Silvio Savarese, Caiming Xiong, Juan Carlos Niebles

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV

MultiMath: Bridging Visual and Mathematical Reasoning for Large Language Models

Shuai Peng, Di Fu, Liangcai Gao, Xiuqin Zhong, Hongguang Fu, Zhi Tang

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.AI

UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models

Liu Qi, He Yongyi, Lian Defu, Zheng Zhi, Xu Tong, Liu Che, Chen Enhong

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.AI

Comments CIKM 2024. The first two authors contributed equally to this work

MMBench: Is Your Multi-modal Model an All-around Player?

Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, Dahua Lin

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted in ECCV2024 as Oral Presentation

Multi-Modal Dialogue State Tracking for Playing GuessWhich Game

Wei Pang, Ruixue Duan, Jinfu Yang, Ning Li

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.AI

Comments Published at CICAI 2023 (CAAI-A), codes at https://github.com/xubuvd/GuessWhich

Do Vision-Language Foundational models show Robust Visual Perception?

Shivam Chandhok, Pranav Tandon

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments UBC Report

Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales

Minghe Gao, Shuang Chen, Liang Pang, Yuan Yao, Jisheng Dang, Wenqiao Zhang, Juncheng Li, Siliang Tang, Yueting Zhuang, Tat-Seng Chua

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving

Ming Nie, Renyuan Peng, Chunwei Wang, Xinyue Cai, Jianhua Han, Hang Xu, Li Zhang

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments ECCV 2024

Advancing Large Multi-modal Models with Explicit Chain-of-Reasoning and Visual Question Generation

Kohei Uehara, Nabarun Goswami, Hanqin Wang, Toshiaki Baba, Kohtaro Tanaka, Tomohiro Hashimoto, Kai Wang, Rei Ito, Takagi Naoya, Ryo Umagami, Yingyi Wen, Tanachai Anakewat, Tatsuya Harada

专题命中 视觉推理 ：VLM(abstract)；分类 cs.CV

Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models

Xihe Qiu, Haoyu Wang, Xiaoyu Tan, Chao Qu, Yujie Xiong, Yuan Cheng, Yinghui Xu, Wei Chu, Yuan Qi

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI

CoReS: Orchestrating the Dance of Reasoning and Segmentation

Xiaoyi Bao, Siyang Sun, Shuailei Ma, Kecheng Zheng, Yuxin Guo, Guosheng Zhao, Yun Zheng, Xingang Wang

专题命中 视觉推理 ：MLLM(abstract)；分类 cs.CV

Comments Accepted at ECCV 2024

VideoCoT: A Video Chain-of-Thought Dataset with Active Annotation Tool

Yan Wang, Yawen Zeng, Jingsheng Zheng, Xiaofen Xing, Jin Xu, Xiangmin Xu

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

Comments ACL 2024 Workshop

ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos

Jr-Jen Chen, Yu-Chien Liao, Hsi-Che Lin, Yu-Chu Yu, Yen-Chun Chen, Yu-Chiang Frank Wang

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://rextime.github.io/

ALGO: Object-Grounded Visual Commonsense Reasoning for Open-World Egocentric Action Recognition

Sanjoy Kundu, Shubham Trehan, Sathyanarayanan N. Aakur

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments Extended abstract of arXiv:2305.16602 for CVPR EgoVis Workshop

GPT4SGG: Synthesizing Scene Graphs from Holistic and Region-specific Narratives

Zuyao Chen, Jinlin Wu, Zhen Lei, Zhaoxiang Zhang, Changwen Chen

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV

Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning

Cheng Tan, Jingxuan Wei, Linzhuang Sun, Zhangyang Gao, Siyuan Li, Bihui Yu, Ruifeng Guo, Stan Z. Li

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments Under review

DecomCAM: Advancing Beyond Saliency Maps through Decomposition and Integration

Yuguang Yang, Runtang Guo, Sheng Wu, Yimi Wang, Linlin Yang, Bo Fan, Jilong Zhong, Juan Zhang, Baochang Zhang

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by Neurocomputing journal

Do Vision-Language Transformers Exhibit Visual Commonsense? An Empirical Study of VCR

Zhenyang Li, Yangyang Guo, Kejie Wang, Xiaolin Chen, Liqiang Nie, Mohan Kankanhalli

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV

TIE: Revolutionizing Text-based Image Editing for Complex-Prompt Following and High-Fidelity Editing

Xinyu Zhang, Mengxue Kang, Fei Wei, Shuang Xu, Yuhe Liu, Lin Ma

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model

Zheng Gu, Shiyuan Yang, Jing Liao, Jing Huo, Yang Gao

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV

Comments Project page: https://analogist2d.github.io