视觉大模型 / VLM

Progressive Multimodal Reasoning via Active Retrieval

Guanting Dong, Chenghao Zhang, Mengjie Deng, Yutao Zhu, Zhicheng Dou, Ji-Rong Wen

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Working in progress

InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions

Pan Zhang, Xiaoyi Dong, Yuhang Cao, Yuhang Zang, Rui Qian, Xilin Wei, Lin Chen, Yifei Li, Junbo Niu, Shuangrui Ding, Qipeng Guo, Haodong Duan, Xin Chen, Han Lv, Zheng Nie, Min Zhang, Bin Wang, Wenwei Zhang, Xinyue Zhang, Jiaye Ge, Wei Li, Jingwen Li, Zhongying Tu, Conghui He, Xingcheng Zhang, Kai Chen, Yu Qiao, Dahua Lin, Jiaqi Wang

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Github Repo: https://github.com/InternLM/InternLM-XComposer/tree/main/InternLM-XComposer-2.5-OmniLive

Towards Neuro-Symbolic Video Understanding

Minkyu Choi, Harsh Goel, Mohammad Omama, Yunhao Yang, Sahil Shah, Sandeep Chinchali

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by The European Conference on Computer Vision (ECCV) 2024

Recurrent Aggregators in Neural Algorithmic Reasoning

Kaijia Xu, Petar Veličković

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Presented at the Third Learning on Graphs Conference (LoG 2024). 10 pages, 1 figure

COLD: Causal reasOning in cLosed Daily activities

Abhinav Joshi, Areeb Ahmad, Ashutosh Modi

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Paper accepted at NeurIPS 2024; Total 37 Pages

Generating Out-Of-Distribution Scenarios Using Language Models

Erfan Aasi, Phat Nguyen, Shiva Sreeram, Guy Rosman, Sertac Karaman, Daniela Rus

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.LG

Multimodal Foundation Models Exploit Text to Make Medical Image Predictions

Thomas Buckley, James A. Diao, Pranav Rajpurkar, Adam Rodman, Arjun K. Manrai

专题命中 视觉推理 ：LLaVA(abstract)；分类 cs.CV、cs.AI

Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects

Michael A. Lepori, Alexa R. Tartaglini, Wai Keen Vong, Thomas Serre, Brenden M. Lake, Ellie Pavlick

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection

Songhao Han, Wei Huang, Hairong Shi, Le Zhuo, Xiu Su, Shifeng Zhang, Xu Zhou, Xiaojuan Qi, Yue Liao, Si Liu

专题命中 视觉推理 ：vision language model(abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 14 figures

XMask3D: Cross-modal Mask Reasoning for Open Vocabulary 3D Semantic Segmentation

Ziyi Wang, Yanbo Wang, Xumin Yu, Jie Zhou, Jiwen Lu

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024

Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination

Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, Lichao Sun

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Multi-modal Situated Reasoning in 3D Scenes

Xiongkun Linghu, Jiangyong Huang, Xuesong Niu, Xiaojian Ma, Baoxiong Jia, Siyuan Huang

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024 Datasets and Benchmarks Track. Project page: https://msr3d.github.io/

ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation

Wenlong Huang, Chen Wang, Yunzhu Li, Ruohan Zhang, Li Fei-Fei

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Bootstrapping Top-down Information for Self-modulating Slot Attention

Dongwon Kim, Seoyeon Kim, Suha Kwak

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.LG

Comments Accepted to NeurIPS 2024

Untrained neural networks can demonstrate memorization-independent abstract reasoning

Tomer Barak, Yonatan Loewenstein

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.AI

Journal ref Sci Rep 14, 27249 (2024)

Neural Concept Binder

Wolfgang Stammer, Antonia Wüst, David Steinmann, Kristian Kersting

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.AI、cs.LG

GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs

Navid Rajabi, Jana Kosecka

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.CV、cs.LG

Comments Accepted to NeurIPS 2024 Workshop on Compositional Learning

Agent S: An Open Agentic Framework that Uses Computers Like a Human

Saaket Agashe, Jiuzhou Han, Shuyu Gan, Jiachen Yang, Ang Li, Xin Eric Wang

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments 23 pages, 16 figures, 9 tables

Needle In A Multimodal Haystack

Weiyun Wang, Shuibo Zhang, Yiming Ren, Yuchen Duan, Tiantong Li, Shuo Liu, Mengkang Hu, Zhe Chen, Kaipeng Zhang, Lewei Lu, Xizhou Zhu, Ping Luo, Yu Qiao, Jifeng Dai, Wenqi Shao, Wenhai Wang

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2024 Track Datasets and Benchmarks

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit

Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Xianglong Liu, Dacheng Tao

专题命中 视觉推理 ：VLM(abstract)；分类 cs.AI、cs.LG

Comments Accepted by EMNLP 2024 Industry Track

One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos

Zechen Bai, Tong He, Haiyang Mei, Pichao Wang, Ziteng Gao, Joya Chen, Lei Liu, Zheng Zhang, Mike Zheng Shou

专题命中 视觉推理 ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurlPS 2024

Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Models

Mike Zhang, Kaixian Qu, Vaishakh Patil, Cesar Cadena, Marco Hutter

专题命中 视觉推理 ：grounding(abstract)；分类 cs.CV、cs.AI

Agentic Skill Discovery

Xufeng Zhao, Cornelius Weber, Stefan Wermter

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments Webpage see https://agentic-skill-discovery.github.io/

Compromising Embodied Agents with Contextual Backdoor Attacks

Aishan Liu, Yuguang Zhou, Xianglong Liu, Tianyuan Zhang, Siyuan Liang, Jiakai Wang, Yanjun Pu, Tianlin Li, Junqi Zhang, Wenbo Zhou, Qing Guo, Dacheng Tao

专题命中 视觉推理 ：visual reasoning(abstract)；分类 cs.AI、cs.LG

GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluation

Yi Zong, Xipeng Qiu

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

ReCorD: Reasoning and Correcting Diffusion for HOI Generation

Jian-Yu Jiang-Lin, Kang-Yang Huang, Ling Lo, Yi-Ning Huang, Terence Lin, Jhih-Ciang Wu, Hong-Han Shuai, Wen-Huang Cheng

专题命中 视觉推理 ：visual language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ACM MM 2024. Project website: https://alberthkyhky.github.io/ReCorD/

Enhancing Human-Centered Dynamic Scene Understanding via Multiple LLMs Collaborated Reasoning

Hang Zhang, Wenxiao Zhang, Haoxuan Qu, Jun Liu

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Towards Temporal Change Explanations from Bi-Temporal Satellite Images

Ryo Tsujimoto, Hiroki Ouchi, Hidetaka Kamigaito, Taro Watanabe

专题命中 视觉推理 ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 6 figures

Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors

Nico Daheim, Jakub Macina, Manu Kapur, Iryna Gurevych, Mrinmaya Sachan

专题命中 视觉推理 ：grounding(abstract)；分类 cs.AI、cs.LG

Comments Preprint. Nico Daheim and Jakub Macina contributed equally. Code and dataset can be found under: https://github.com/eth-lre/verify-then-generate

Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning

Bingchen Zhao, Yongshuo Zong, Letian Zhang, Timothy Hospedales

专题命中 视觉推理 ：visual language model(abstract)；分类 cs.CV、cs.AI

Comments First three authors contributed equally. Dataset: https://huggingface.co/datasets/VLLMs/MIRB