视觉大模型 / VLM

A Divide-Align-Conquer Strategy for Program Synthesis

Jonas Witt, Sebastijan Dumančić, Tias Guns, Claus-Christian Carbon

机构 * University of Bamberg（班贝格大学） ； TU Delft（代尔夫特理工大学） ； KU Leuven（鲁汶大学） 纠错

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.AI

Comments 11 pages, 9 figures

Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval

Yuanmin Tang, Xiaoting Qin, Jue Zhang, Jing Yu, Gaopeng Gou, Gang Xiong, Qingwei Ling, Saravan Rajmohan, Dongmei Zhang, Qi Wu

机构 * Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所） ； University of Chinese Academy of Sciences（中国科学院大学） ； Microsoft（微软公司） ； Minzu University of China（中央民族大学） ； University of Adelaide（阿德莱德大学） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

An Agentic Approach to Automatic Creation of P&ID Diagrams from Natural Language Descriptions

Shreeyash Gowaikar, Srinivasan Iyengar, Sameer Segal, Shivkumar Kalyanaraman

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.LG

Comments Accepted at the AAAI'25 Workshop on AI to Accelerate Science and Engineering (AI2ASE)

Leveraging Retrieval-Augmented Tags for Large Vision-Language Understanding in Complex Scenes

Antonio Carlos Rivera, Anthony Moore, Steven Robinson

机构 * EDP University of Puerto Rico（波多黎各EDP大学） 纠错

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Benchmarking VLMs' Reasoning About Persuasive Atypical Images

Sina Malakouti, Aysan Aghazadeh, Ashmit Khandelwal, Adriana Kovashka

机构 * University of Pittsburgh（匹兹堡大学） ； BITS Pilani（pilani工学院） 纠错

专题命中 视觉推理 ：vision language model(abstract)；分类 cs.CV

Piecing It All Together: Verifying Multi-Hop Multimodal Claims

Haoran Wang, Aman Rangapur, Xiongxiao Xu, Yueqing Liang, Haroon Gharwi, Carl Yang, Kai Shu

机构 * Illinois Institute of Technology（伊利诺伊理工学院） ； Emory University（埃默里大学） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.AI

Comments COLING 2025

Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs

Rong Wang, Kun Sun, Jonas Kuhn

机构 * Stuttgart University（斯图加特大学） 纠错

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI

R2G: Reasoning to Ground in 3D Scenes

Yixuan Li, Zan Wang, Wei Liang

机构 * Beijing Institute of Technology（北京理工大学） ； Yangtze Delta Region Academy of Beijing Institute of Technology（北京理工大学长三角研究院） 纠错

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV

EditScout: Locating Forged Regions from Diffusion-based Edited Images with Multimodal LLM

Quang Nguyen, Truong Vu, Trong-Tung Nguyen, Yuxin Wen, Preston K Robinette, Taylor T Johnson, Tom Goldstein, Anh Tran, Khoi Nguyen

机构 * VinAI Research（VinAI研究院） ； University of Maryland（马里兰大学） ； Vanderbilt University（范德堡大学） 纠错

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

Improving Multimodal LLMs Ability In Geometry Problem Solving, Reasoning, And Multistep Scoring

Avinash Anand, Raj Jaiswal, Abhishek Dharmadhikari, Atharva Marathe, Harsh Parimal Popat, Harshil Mital, Kritarth Prasad, Rajiv Ratn Shah, Roger Zimmermann

专题命中 视觉推理 ：vision language model(abstract)；分类 cs.AI

Comments 15 pages

VisScience: An Extensive Benchmark for Evaluating K12 Educational Multi-modal Scientific Reasoning

Zhihuan Jiang, Zhen Yang, Jinhao Chen, Zhengxiao Du, Weihan Wang, Bin Xu, Jie Tang

专题命中 视觉推理 ：visual question answering(abstract)；分类 cs.AI

Comments 89 pages, 70 figures

Slot State Space Models

Jindong Jiang, Fei Deng, Gautam Singh, Minseung Lee, Sungjin Ahn

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.AI

Comments Accepted to NeurIPS 2024; Project page is available at https://slotssms.github.io/ ; Code is available at https://github.com/JindongJiang/SlotSSMs

3D Scene Graph Guided Vision-Language Pre-training

Hao Liu, Yanni Ma, Yan Liu, Haihong Xiao, Ying He

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV

Comments 14 pages, 8 figures, 7 tables

Abductive Symbolic Solver on Abstraction and Reasoning Corpus

Mintaek Lim, Seokki Lee, Liyew Woletemaryam Abitew, Sundong Kim

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.AI

Comments Presented at IJCAI 2024 LNSAI Workshop

NEMO: Can Multimodal LLMs Identify Attribute-Modified Objects?

Jiaxuan Li, Junwen Mo, MinhDuc Vo, Akihiro Sugimoto, Hideki Nakayama

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

Decompose and Leverage Preferences from Expert Models for Improving Trustworthiness of MLLMs

Rui Cao, Yuming Jiang, Michael Schlichtkrull, Andreas Vlachos

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV

ConMe: Rethinking Evaluation of Compositional Reasoning for Modern VLMs

Irene Huang, Wei Lin, M. Jehanzeb Mirza, Jacob A. Hansen, Sivan Doveh, Victor Ion Butoi, Roei Herzig, Assaf Arbelle, Hilde Kuehne, Trevor Darrell, Chuang Gan, Aude Oliva, Rogerio Feris, Leonid Karlinsky

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS 2024 Camera Ready

Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models

Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, Ranjay Krishna

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2024. Project and codes url: https://visualsketchpad.github.io/

Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning

Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Jingdong Sun, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, Alexander Hauptmann

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.AI

Comments Accepted at NeurIPS 2024. 49 pages, 13 figures, Project: https://github.com/ZebangCheng/Emotion-LLaMA, Demo: https://huggingface.co/spaces/ZebangCheng/Emotion-LLaMA

PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks

Matthew Chang, Gunjan Chhablani, Alexander Clegg, Mikael Dallaire Cote, Ruta Desai, Michal Hlavac, Vladimir Karashchuk, Jacob Krantz, Roozbeh Mottaghi, Priyam Parashar, Siddharth Patki, Ishita Prasad, Xavier Puig, Akshara Rai, Ram Ramrakhya, Daniel Tran, Joanne Truong, John M. Turner, Eric Undersander, Tsung-Yen Yang

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI

Comments Alphabetical author order

MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding

Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, Kai Chen

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments Accepted in NeurIPS 2024 Datasets and Benchmarks Track

Code Repair with LLMs gives an Exploration-Exploitation Tradeoff

Hao Tang, Keya Hu, Jin Peng Zhou, Sicheng Zhong, Wei-Long Zheng, Xujie Si, Kevin Ellis

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.AI

ImageInWords: Unlocking Hyper-Detailed Image Descriptions

Roopal Garg, Andrea Burns, Burcu Karagol Ayan, Yonatan Bitton, Ceslee Montgomery, Yasumasa Onoe, Andrew Bunner, Ranjay Krishna, Jason Baldridge, Radu Soricut

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments Webpage (https://google.github.io/imageinwords), GitHub (https://github.com/google/imageinwords), HuggingFace (https://huggingface.co/datasets/google/imageinwords)

Beyond Color and Lines: Zero-Shot Style-Specific Image Variations with Coordinated Semantics

Jinghao Hu, Yuhe Zhang, GuoHua Geng, Liuyuxin Yang, JiaRui Yan, Jingtao Cheng, YaDong Zhang, Kang Li

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments 13 pages,6 figures

Scene Graph Generation with Role-Playing Large Language Models

Guikun Chen, Jin Li, Wenguan Wang

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments NeurIPS 2024. Code: https://github.com/guikunchen/SDSGG

Differentiable Robot Rendering

Ruoshi Liu, Alper Canberk, Shuran Song, Carl Vondrick

专题命中 视觉推理 ：vision language model(abstract)；分类 cs.CV

Comments Project Page: https://drrobot.cs.columbia.edu/

VL-GLUE: A Suite of Fundamental yet Challenging Visuo-Linguistic Reasoning Tasks

Shailaja Keyur Sampat, Mutsumi Nakamura, Shankar Kailas, Kartik Aggarwal, Mandy Zhou, Yezhou Yang, Chitta Baral

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV

Comments 18 pages, 7 figures

Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptions

Zhenyu Jiang, Yuqi Xie, Jinhan Li, Ye Yuan, Yifeng Zhu, Yuke Zhu

专题命中 视觉推理 ：vision language model(abstract)；分类 cs.AI

Comments Accepted for oral presentation at 8th Annual Conference on Robot Learning. Project website: https://ut-austin-rpl.github.io/Harmon/

MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark

Bin Shan, Xiang Fei, Wei Shi, An-Lan Wang, Guozhi Tang, Lei Liao, Jingqun Tang, Xiang Bai, Can Huang

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV

Comments 12 pages, 5 figures, project page: https://github.com/xfey/MCTBench?tab=readme-ov-file

Structured Spatial Reasoning with Open Vocabulary Object Detectors

Negar Nejatishahidin, Madhukar Reddy Vongala, Jana Kosecka

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV