多模态大模型

VideoXum: Cross-modal Visual and Textural Summarization of Videos

Jingyang Lin, Hang Hua, Ming Chen, Yikang Li, Jenhao Hsiao, Chiuman Ho, Jiebo Luo

专题命中 多模态评测 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments 13 pages, 7 figures

Journal ref IEEE Transactions on Multimedia, VOL. 26 (2024) 5548-5560

Iteratively Prompting Multimodal LLMs to Reproduce Natural and AI-Generated Images

Ali Naseh, Katherine Thai, Mohit Iyyer, Amir Houmansadr

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL

Towards Efficient Resume Understanding: A Multi-Granularity Multi-Modal Pre-Training Approach

Feihu Jiang, Chuan Qin, Jingshuai Zhang, Kaichun Yao, Xi Chen, Dazhong Shen, Chen Zhu, Hengshu Zhu, Hui Xiong

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments ICME 2024 Accepted

MultiCorrupt: A Multi-Modal Robustness Dataset and Benchmark of LiDAR-Camera Fusion for 3D Object Detection

Till Beemelmanns, Quan Zhang, Christian Geller, Lutz Eckstein

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Code: https://github.com/ika-rwth-aachen/MultiCorrupt

What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models

Letian Zhang, Xiaotong Zhai, Zhongkai Zhao, Yongshuo Zong, Xin Wen, Bingchen Zhao

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning

Fuxiao Liu, Xiaoyang Wang, Wenlin Yao, Jianshu Chen, Kaiqiang Song, Sangwoo Cho, Yaser Yacoob, Dong Yu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted to NAACL 2024

MM-PhyQA: Multimodal Physics Question-Answering With Multi-Image CoT Prompting

Avinash Anand, Janak Kapuriya, Apoorv Singh, Jay Saraf, Naman Lal, Astha Verma, Rushali Gupta, Rajiv Shah

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding?

Junpeng Liu, Yifan Song, Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li, Xiang Yue

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Interpretable Multimodal Learning for Cardiovascular Hemodynamics Assessment

Prasun C Tripathi, Sina Tabakhi, Mohammod N I Suvon, Lawrence Schöb, Samer Alabed, Andrew J Swift, Shuo Zhou, Haiping Lu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

TinyVQA: Compact Multimodal Deep Neural Network for Visual Question Answering on Resource-Constrained Devices

Hasib-Al Rashid, Argho Sarkar, Aryya Gangopadhyay, Maryam Rahnemoonfar, Tinoosh Mohsenin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted as a full paper by the tinyML Research Symposium 2024

SCANNER: Knowledge-Enhanced Approach for Robust Multi-modal Named Entity Recognition of Unseen Entities

Hyunjong Ok, Taeho Kil, Sukmin Seo, Jaeho Lee

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 13 pages, 7 figures, NAACL 2024

MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements

Lisong C. Sun, Neel P. Bhatt, Jonathan C. Liu, Zhiwen Fan, Zhangyang Wang, Todd E. Humphreys, Ufuk Topcu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Project Webpage: https://vita-group.github.io/MM3DGS-SLAM

Neural Multimodal Topic Modeling: A Comprehensive Evaluation

Felipe González-Pizarro, Giuseppe Carenini

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Camera-Ready for LREC-COLING 2024 (Long Paper)

MOGAM: A Multimodal Object-oriented Graph Attention Model for Depression Detection

Junyeop Cha, Seoyun Kim, Dongjae Kim, Eunil Park

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 3 figures, 4 tables

A Unified Model for Longitudinal Multi-Modal Multi-View Prediction with Missingness

Boqi Chen, Junier Oliva, Marc Niethammer

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Aria-NeRF: Multimodal Egocentric View Synthesis

Jiankai Sun, Jianing Qiu, Chuanyang Zheng, John Tucker, Javier Yu, Mac Schwager

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models

Rocktim Jyoti Das, Simeon Emilov Hristov, Haonan Li, Dimitar Iliyanov Dimitrov, Ivan Koychev, Preslav Nakov

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document

Yuliang Liu, Biao Yang, Qiang Liu, Zhang Li, Zhiyin Ma, Shuo Zhang, Xiang Bai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Are Language Models Puzzle Prodigies? Algorithmic Puzzles Unveil Serious Challenges in Multimodal Reasoning

Deepanway Ghosal, Vernon Toh Yan Han, Chia Yew Ken, Soujanya Poria

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Pink: Unveiling the Power of Referential Comprehension for Multi-modal LLMs

Shiyu Xuan, Qingpei Guo, Ming Yang, Shiliang Zhang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

MMAPS: End-to-End Multi-Grained Multi-Modal Attribute-Aware Product Summarization

Tao Chen, Ze Lin, Hui Li, Jiayi Ji, Yiyi Zhou, Guanbin Li, Rongrong Ji

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.MM

Comments LREC-COLING 2024.11 pages, 4 figures

PALO: A Polyglot Large Multimodal Model for 5B People

Muhammad Maaz, Hanoona Rasheed, Abdelrahman Shaker, Salman Khan, Hisham Cholakal, Rao M. Anwer, Tim Baldwin, Michael Felsberg, Fahad S. Khan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Technical Report of PALO

GroundingGPT:Language Enhanced Multi-modal Grounding Model

Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, Yiqing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Van Tu Vu, Zhida Huang, Tao Wang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Modality-Aware and Shift Mixer for Multi-modal Brain Tumor Segmentation

Zhongzhen Huang, Linda Wei, Shaoting Zhang, Xiaofan Zhang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

REWIND Dataset: Privacy-preserving Speaking Status Segmentation from Multimodal Body Movement Signals in the Wild

Jose Vargas Quiros, Chirag Raman, Stephanie Tan, Ekin Gedik, Laura Cabrera-Quiros, Hayley Hung

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Exploring Multimodal Large Language Models for Radiology Report Error-checking

Jinge Wu, Yunsoo Kim, Eva C. Keller, Jamie Chow, Adam P. Levine, Nikolas Pontikos, Zina Ibrahim, Paul Taylor, Michelle C. Williams, Honghan Wu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey

Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, Jiaqi Li, Xiaoze Liu, Jeff Z. Pan, Ningyu Zhang, Huajun Chen

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Ongoing work; 41 pages (Main Text), 55 pages (Total), 11 Tables, 13 Figures, 619 citations; Paper list is available at https://github.com/zjukg/KG-MM-Survey

MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing

Jiaqi Li, Miaozeng Du, Chuanyi Zhang, Yongrui Chen, Nan Hu, Guilin Qi, Haiyun Jiang, Siyuan Cheng, Bozhong Tian

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 8 pages

Text Role Classification in Scientific Charts Using Multimodal Transformers

Hye Jin Kim, Nicolas Lell, Ansgar Scherp

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Exploring Perceptual Limitation of Multimodal Large Language Models

Jiarui Zhang, Jinyi Hu, Mahyar Khayatkhoei, Filip Ilievski, Maosong Sun

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 14 figures, 3 tables