视觉大模型 / VLM

ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification

Ziqing Fan, Cheng Liang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, Weidi Xie

机构 * Shanghai Jiao Tong University（上海交通大学） ； Shanghai Artificial Intelligence Laboratory（上海人工智能实验室） 纠错

专题命中 视觉推理 ：visual question answering(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

A Cognitive Paradigm Approach to Probe the Perception-Reasoning Interface in VLMs

Mohit Vaishnav, Tanel Tammet

机构 * Applied Artificial Intelligence Group, Tallinn University of Technology, Estonia（塔林技术大学应用人工智能组） ； Kimova AI, Tallinn, Estonia（Kimova AI） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；visual reasoning(abstract)；分类 cs.CV、cs.AI

TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action

Jen-Hao Cheng, Vivian Wang, Huayu Wang, Huapeng Zhou, Yi-Hao Peng, Hou-I Liu, Hsiang-Wei Huang, Kuang-Ming Chen, Cheng-Yen Yang, Wenhao Chai, Yi-Ling Chen, Vibhav Vineet, Qin Cai, Jenq-Neng Hwang

机构 * University of Washington（华盛顿大学） ； Carnegie Mellon University（卡内基梅隆大学） ； National Yang Ming Chiao Tung University ； Microsoft（微软公司） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning

Baining Zhao, Ziyou Wang, Jianjie Fang, Chen Gao, Fanhang Man, Jinqiang Cui, Xin Wang, Xinlei Chen, Yong Li, Wenwu Zhu

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 5 figures

Look Before You Decide: Prompting Active Deduction of MLLMs for Assumptive Reasoning

Yian Li, Wentao Tian, Yang Jiao, Jingjing Chen, Tianwen Qian, Bin Zhu, Na Zhao, Yu-Gang Jiang

专题命中 视觉推理 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Black Swan: Abductive and Defeasible Video Reasoning in Unpredictable Events

Aditya Chinchure, Sahithya Ravi, Raymond Ng, Vered Shwartz, Boyang Li, Leonid Sigal

专题命中 视觉推理 ：vision-language model(abstract)；LLaVA(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2025. For data, visit https://blackswan.cs.ubc.ca

VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Prior

Xindi Yang, Baolu Li, Yiming Zhang, Zhenfei Yin, Lei Bai, Liqian Ma, Zhiyong Wang, Jianfei Cai, Tien-Tsin Wong, Huchuan Lu, Xu Jia

专题命中 视觉推理 ：vision language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 18 pages, 11 figures

Sightation Counts: Leveraging Sighted User Feedback in Building a BLV-aligned Dataset of Diagram Descriptions

Wan Ju Kang, Eunki Kim, Na Min An, Sangryul Kim, Haemin Choi, Ki Hoon Kwak, James Thorne

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 37 pages, 10 figures, 21 tables

EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks

Yi Zhang, Qiang Zhang, Xiaozhu Ju, Zhaoyang Liu, Jilei Mao, Jingkai Sun, Jintao Wu, Shixiong Gao, Shihan Cai, Zhiyuan Qin, Linkai Liang, Jiaxu Wang, Yiqun Duan, Jiahang Cao, Renjing Xu, Jian Tang

专题命中 视觉推理 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments technical report

Enhancing Cognition and Explainability of Multimodal Foundation Models with Self-Synthesized Data

Yucheng Shi, Quanzheng Li, Jin Sun, Xiang Li, Ninghao Liu

专题命中 视觉推理 ：vision-language model(abstract)；visual reasoning(abstract)；分类 cs.CV、cs.LG

Comments Accepted by ICLR 2025. Code: https://github.com/sycny/SelfSynthX

Efficient Reasoning with Hidden Thinking

Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, Jiuxiang Gu

专题命中 视觉推理 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.AI、cs.LG

Comments Preprint version

SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning

Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao, Guowei Huang, Helong Huang, Guangjian Tian, Weichao Qiu, Xingyue Quan, Jianye Hao, Yuzheng Zhuang

专题命中 视觉推理 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments Under Review

JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images

Zhecan Wang, Junzhang Liu, Chia-Wei Tang, Hani Alomari, Anushka Sivakumar, Rui Sun, Wenhao Li, Md. Atabuzzaman, Hammad Ayyubi, Haoxuan You, Alvi Ishmam, Kai-Wei Chang, Shih-Fu Chang, Chris Thomas

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Socratic Questioning: Learn to Self-guide Multimodal Reasoning in the Wild

Wanpeng Hu, Haodi Liu, Lin Chen, Feng Zhou, Changming Xiao, Qi Yang, Changshui Zhang

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

PhotoBot: Reference-Guided Interactive Photography via Natural Language

Oliver Limoyo, Jimmy Li, Dmitriy Rivkin, Jonathan Kelly, Gregory Dudek

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI

Comments In Proceedings of the IEEE/RSJ International Conference on Intelligent Robotics and Systems (IROS'24), Abu Dhabi, UAE, Oct. 14-18, 2024

Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning

Qi Sun, Pengfei Hong, Tej Deep Pala, Vernon Toh, U-Xuan Tan, Deepanway Ghosal, Soujanya Poria

专题命中 视觉推理 ：visual language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

Comments https://github.com/declare-lab/Emma-X, https://huggingface.co/declare-lab/Emma-X

NL-Eye: Abductive NLI for Images

Mor Ventura, Michael Toker, Nitay Calderon, Zorik Gekhman, Yonatan Bitton, Roi Reichart

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI

How Far Are We from Intelligent Visual Deductive Reasoning?

Yizhe Zhang, He Bai, Ruixiang Zhang, Jiatao Gu, Shuangfei Zhai, Josh Susskind, Navdeep Jaitly

专题命中 视觉推理 ：vision-language model(abstract)；visual reasoning(abstract)；分类 cs.CV、cs.AI

Comments COLM 2024. https://github.com/apple/ml-rpm-bench

CadVLM: Bridging Language and Vision in the Generation of Parametric CAD Sketches

Sifan Wu, Amir Khasahmadi, Mor Katz, Pradeep Kumar Jayaraman, Yewen Pu, Karl Willis, Bang Liu

专题命中 视觉推理 ：vision language model(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

NEUSIS: A Compositional Neuro-Symbolic Framework for Autonomous Perception, Reasoning, and Planning in Complex UAV Search Missions

Zhixi Cai, Cristian Rojas Cardenas, Kevin Leo, Chenyuan Zhang, Kal Backman, Hanbing Li, Boying Li, Mahsa Ghorbanali, Stavya Datta, Lizhen Qu, Julian Gutierrez Santiago, Alexey Ignatiev, Yuan-Fang Li, Mor Vered, Peter J Stuckey, Maria Garcia de la Banda, Hamid Rezatofighi

专题命中 视觉推理 ：vision-language model(abstract)；grounding(abstract)；分类 cs.CV、cs.AI

HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation

Jin Wang, Rui Dai, Weijie Wang, Luca Rossini, Francesco Ruscelli, Nikos Tsagarakis

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.AI、cs.LG

Comments Project page: https://hy-motion.github.io/

X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs

Sirnam Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, Mubarak Shah

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted at ECCV2024

Whiteboard-of-Thought: Thinking Step-by-Step Across Modalities

Sachit Menon, Richard Zemel, Carl Vondrick

专题命中 视觉推理 ：visual reasoning(abstract)；multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Project website: whiteboard.cs.columbia.edu/

RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understanding

Linrui Xu, Ling Zhao, Wang Guo, Qiujun Li, Kewang Long, Kaiqi Zou, Yuhan Wang, Haifeng Li

专题命中 视觉推理 ：visual reasoning(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 6 figures, 4 tables

Conceptual Learning via Embedding Approximations for Reinforcing Interpretability and Transparency

Maor Dikter, Tsachi Blau, Chaim Baskin

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.LG

Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation data

Chenhui Zhang, Sherrie Wang

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments 62 pages; work in progress

A Challenger to GPT-4V? Early Explorations of Gemini in Visual Expertise

Chaoyou Fu, Renrui Zhang, Zihan Wang, Yubo Huang, Zhengye Zhang, Longtian Qiu, Gaoxiang Ye, Yunhang Shen, Mengdan Zhang, Peixian Chen, Sirui Zhao, Shaohui Lin, Deqiang Jiang, Di Yin, Peng Gao, Ke Li, Hongsheng Li, Xing Sun

专题命中 视觉推理 ：visual reasoning(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments Total 120 pages. See our project at https://github.com/BradyFU/Awesome-Multimodal-Large-Language-Models

Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond

Liang Chen, Yichi Zhang, Shuhuai Ren, Haozhe Zhao, Zefan Cai, Yuchi Wang, Peiyi Wang, Tianyu Liu, Baobao Chang

专题命中 视觉推理 ：multimodal large language model(abstract)；MLLM(abstract)；分类 cs.CV、cs.AI

Comments FMDM@NeurIPS2023, Code and data: https://github.com/pkunlp-icler/PCA-EVAL/

ViLLA: Fine-Grained Vision-Language Representation Learning from Real-World Data

Maya Varma, Jean-Benoit Delbrouck, Sarah Hooper, Akshay Chaudhari, Curtis Langlotz

专题命中 视觉推理 ：vision-language model(abstract)；VLM(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023

Preconditioned Visual Language Inference with Weak Supervision

Ehsan Qasemi, Amani R. Maina-Kilaas, Devadutta Dash, Khalid Alsaggaf, Muhao Chen

专题命中 视觉推理 ：VLM(abstract)；visual language model(abstract)；分类 cs.CV、cs.AI