多模态大模型

Effective Backdoor Mitigation in Vision-Language Models Depends on the Pre-training Objective

Sahil Verma, Gantavya Bhatt, Avi Schwarzschild, Soumye Singhal, Arnav Mohanty Das, Chirag Shah, John P Dickerson, Pin-Yu Chen, Jeff Bilmes

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at TMLR (https://openreview.net/forum?id=Conma3qnaT)

From Simple to Professional: A Combinatorial Controllable Image Captioning Agent

Xinran Wang, Muxi Diao, Baoteng Li, Haiwen Zhang, Kongming Liang, Zhanyu Ma

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments A technical report. Project: https://github.com/xin-ran-w/CapAgent

JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images

Zhecan Wang, Junzhang Liu, Chia-Wei Tang, Hani Alomari, Anushka Sivakumar, Rui Sun, Wenhao Li, Md. Atabuzzaman, Hammad Ayyubi, Haoxuan You, Alvi Ishmam, Kai-Wei Chang, Shih-Fu Chang, Chris Thomas

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding

Xingyu Fu, Minqian Liu, Zhengyuan Yang, John Corring, Yijuan Lu, Jianwei Yang, Dan Roth, Dinei Florencio, Cha Zhang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Project link: https://zeyofu.github.io/ReFocus/

Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understanding

Aaron Lohner, Francesco Compagno, Jonathan Francis, Alessandro Oltramari

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Won the 'Best Paper Runner-up Award' at the 2024 IEEE International Automated Vehicle Validation Conference (IAVVC 2024). Also accepted at the 1st Workshop on Semantic Reasoning and Goal Understanding in Robotics, at the Robotics Science and Systems Conference (RSS SemRob 2024)

Unified Coding for Both Human Perception and Generalized Machine Analytics with CLIP Supervision

Kangsheng Yin, Quan Liu, Xuelin Shen, Yulin He, Wenhan Yang, Shiqi Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 9 pages, 10 figures, publised to AAAI 2025

Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches

Alhassan Mumuni, Fuseini Mumuni

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

CRRG-CLIP: Automatic Generation of Chest Radiology Reports and Classification of Chest Radiographs

Jianfei Xu, Thanet Markchom, Huizhi Liang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

ECG-guided individual identification via PPG

Riling Wei, Hanjie Chen, Kelu Yao, Chuanguang Yang, Jun Wang, Chao Li

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICASSP 2025. Camera Ready Version

Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue

Kun Ouyang, Liqiang Jing, Xuemeng Song, Meng Liu, Yupeng Hu, Liqiang Nie

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.MM

Comments This paper got accepted by IEEE TMM

ChartAdapter: Large Vision-Language Model for Chart Summarization

Peixin Xu, Yujuan Ding, Wenqi Fan

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CL、cs.MM

Tuning Vision-Language Models with Candidate Labels by Prompt Alignment

Zhifang Zhang, Yuwei Niu, Xin Liu, Beibei Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Referencing Where to Focus: Improving VisualGrounding with Referential Query

Yabing Wang, Zhuotao Tian, Qingpei Guo, Zheng Qin, Sanping Zhou, Ming Yang, Le Wang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by NIPS2024

Open-Vocabulary Panoptic Segmentation Using BERT Pre-Training of Vision-Language Multiway Transformer Model

Yi-Chia Chen, Wei-Hua Li, Chu-Song Chen

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments ICIP 2024

HiRED: Attention-Guided Token Dropping for Efficient Inference of High-Resolution Vision-Language Models

Kazi Hasan Ibn Arif, JinYi Yoon, Dimitrios S. Nikolopoulos, Hans Vandierendonck, Deepu John, Bo Ji

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted in AAAI 2025

Progressive Multi-granular Alignments for Grounded Reasoning in Large Vision-Language Models

Quang-Hung Le, Long Hoang Dang, Ngan Le, Truyen Tran, Thao Minh Le

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning

Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Lok Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert, Timothy Rogers, Kevin Jamieson, Robert Mankoff, Robert Nowak

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL、cs.AI

LMM-Regularized CLIP Embeddings for Image Classification

Maria Tzelepi, Vasileios Mezaris

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted for publication, 26th Int. Symp. on Multimedia (IEEE ISM 2024), Tokyo, Japan, Dec. 2024. This is the authors' "accepted version"

Learning from Synthetic Data for Visual Grounding

Ruozhen He, Ziyan Yang, Paola Cascante-Bonilla, Alexander C. Berg, Vicente Ordonez

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Project Page: https://catherine-r-he.github.io/SynGround/

Position-aware Guided Point Cloud Completion with CLIP Model

Feng Zhou, Qi Zhang, Ju Dai, Lei Li, Qing Fan, Junliang Xing

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by AAAI25

Seeing Syntax: Uncovering Syntactic Learning Limitations in Vision-Language Models

Sri Harsha Dumpala, David Arps, Sageev Oore, Laura Kallmeyer, Hassan Sajjad

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Towards Efficient and Robust VQA-NLE Data Generation with Large Vision-Language Models

Patrick Amadeus Irawan, Genta Indra Winata, Samuel Cahyawijaya, Ayu Purwarianti

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Preprint

doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigation

Parthib Roy, Srinivasa Perisetla, Shashank Shriram, Harsha Krishnaswamy, Aryan Keskar, Ross Greer

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

MG-3D: Multi-Grained Knowledge-Enhanced 3D Medical Vision-Language Pre-training

Xuefeng Ni, Linshan Wu, Jiaxin Zhuang, Qiong Wang, Mingxiang Wu, Varut Vardhanabhuti, Lihai Zhang, Hanyu Gao, Hao Chen

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 10 Pages

Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents

Jun Chen, Dannong Xu, Junjie Fei, Chun-Mei Feng, Mohamed Elhoseiny

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments the correct arxiv version

Cross-Self KV Cache Pruning for Efficient Vision-Language Inference

Xiaohuan Pei, Tao Huang, Chang Xu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characters

Jianping Jiang, Weiye Xiao, Zhengyu Lin, Huaizhong Zhang, Tianxiang Ren, Yang Gao, Zhiqian Lin, Zhongang Cai, Lei Yang, Ziwei Liu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV、cs.AI

VARCO-VISION: Expanding Frontiers in Korean Vision-Language Models

Jeongho Ju, Daeyoung Kim, SunYoung Park, Youngjune Kim

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments 24 pages, 15 figures, 4 tables. Model weights at https://huggingface.co/NCSOFT/VARCO-VISION-14B. Benchmarks released at NCSOFT's HuggingFace repositories (K-MMBench, K-SEED, K-MMStar, K-DTCBench, K-LLaVA-W). VARCO-VISION is an open-source Korean-English VLM with OCR, grounding, and referring capabilities

VLM-HOI: Vision Language Models for Interpretable Human-Object Interaction Analysis

Donggoo Kang, Dasol Jeong, Hyunmin Lee, Sangwoo Park, Hasil Park, Sunkyu Kwon, Yeongjoon Kim, Joonki Paik

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments 18 pages

What's in the Image? A Deep-Dive into the Vision of Vision Language Models

Omri Kaduri, Shai Bagon, Tali Dekel

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI