多模态大模型

Friends-MMC: A Dataset for Multi-modal Multi-party Conversation Understanding

Yueqian Wang, Xiaojun Meng, Yuxuan Wang, Jianxin Liang, Qun Liu, Dongyan Zhao

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL

Comments Published at AAAI 2025

Do Multimodal Language Models Really Understand Direction? A Benchmark for Compass Direction Reasoning

Hang Yin, Zhifeng Lin, Xin Liu, Bin Sun, Kan Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

J-EDI QA: Benchmark for deep-sea organism-specific multimodal LLM

Takero Yoshida, Yuikazu Ito, Yoshihiro Fujiwara, Shinji Tsuchida, Daisuke Sugiyama, Daisuke Matsuoka

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

XFormParser: A Simple and Effective Multimodal Multilingual Semi-structured Form Parser

Xianfu Cheng, Hang Zhang, Jian Yang, Xiang Li, Weixiao Zhou, Fei Liu, Kui Wu, Xiangyuan Guan, Tao Sun, Xianjie Wu, Tongliang Li, Zhoujun Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

Comments 15 pages, 8 figures, 8 tables

ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing

Yaohui Ma, Xiaopeng Hong, Shizhou Zhang, Huiyun Li, Zhilin Zhu, Wei Luo, Zhiheng Ma

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Extended version for paper accepted to AAAI 2025. Project Page: https://github.com/yaohui120/ComprehendEdit

Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection

Jiaying Lin, Yuen-Hei Yeung, Shuquan Ye, Rynson W. H. Lau

专题命中 多模态评测 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted to AAAI 2025. Project Page: https://jiaying.link/AAAI25-RGBDGlass/

ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data

Chengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun, Zirui Zhuang, Jinming Wu, Lei Zhang, Jianxin Liao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by AAAI 2025

Unimodal and Multimodal Static Facial Expression Recognition for Virtual Reality Users with EmoHeVRDB

Thorben Ortmann, Qi Wang, Larissa Putzar

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control

Mariam Hassan, Sebastian Stapf, Ahmad Rahimi, Pedro M B Rezende, Yasaman Haghighi, David Brüggemann, Isinsu Katircioglu, Lin Zhang, Xiaoran Chen, Suman Saha, Marco Cannici, Elie Aljalbout, Botao Ye, Xi Wang, Aram Davtyan, Mathieu Salzmann, Davide Scaramuzza, Marc Pollefeys, Paolo Favaro, Alexandre Alahi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

M$^{3}$D: A Multimodal, Multilingual and Multitask Dataset for Grounded Document-level Information Extraction

Jiang Liu, Bobo Li, Xinran Yang, Na Yang, Hao Fei, Mingyao Zhang, Fei Li, Donghong Ji

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

Comments 14 pages, 9 figures, 6 tables

Red Teaming GPT-4V: Are GPT-4V Safe Against Uni/Multi-Modal Jailbreak Attacks?

Shuo Chen, Zhen Han, Bailan He, Zifeng Ding, Wenqian Yu, Philip Torr, Volker Tresp, Jindong Gu

专题命中 多模态评测 ：multi-modal(title)；multimodal(abstract)；分类 cs.CL

Comments technical report; update code repo link

LAION-SG: An Enhanced Large-Scale Dataset for Training Complex Image-Text Models with Structural Annotations

Zejian Li, Chenye Meng, Yize Li, Ling Yang, Shengyuan Zhang, Jiarui Ma, Jiayi Li, Guang Yang, Changyuan Yang, Zhiyuan Yang, Jinxiong Chang, Lingyun Sun

专题命中 多模态评测 ：image-text(title,abstract)；分类 cs.CV

Exemplar Masking for Multimodal Incremental Learning

Yi-Lun Lee, Chen-Yu Lee, Wei-Chen Chiu, Yi-Hsuan Tsai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Project page: https://github.com/YiLunLee/Exemplar_Masking_MCIL

A multimodal ensemble approach for clear cell renal cell carcinoma treatment outcome prediction

Meixu Chen, Kai Wang, Payal Kapur, James Brugarolas, Raquibul Hannan, Jing Wang

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 3 figures, 4 tables

CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy

Zhibo Yang, Jun Tang, Zhaohai Li, Pengfei Wang, Jianqiang Wan, Humen Zhong, Xuejing Liu, Mingkun Yang, Peng Wang, Shuai Bai, LianWen Jin, Junyang Lin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments 23 pages, 14 figures; The code will be released soon

Syn-Mediverse: A Multimodal Synthetic Dataset for Intelligent Scene Understanding of Healthcare Facilities

Rohit Mohan, José Arce, Sassan Mokhtar, Daniele Cattaneo, Abhinav Valada

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Journal ref IEEE Robotics and Automation Letters, vol. 9, no. 8, pp. 7094-7101, Aug. 2024

DevBench: A multimodal developmental benchmark for language learning

Alvin Wei Ming Tan, Sunny Yu, Bria Long, Wanjing Anya Ma, Tonya Murray, Rebecca D. Silverman, Jason D. Yeatman, Michael C. Frank

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at NeurIPS 2024 (Oral)

ActiveAnno3D -- An Active Learning Framework for Multi-Modal 3D Object Detection

Ahmed Ghita, Bjørk Antoniussen, Walter Zimmer, Ross Greer, Christian Creß, Andreas Møgelmose, Mohan M. Trivedi, Alois C. Knoll

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Comments 2024 Proceedings of the IEEE Intelligent Vehicles Symposium 2024 (IV'24)

TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction

Zeynel A. Samak, Philip Clatworthy, Majid Mirmehdi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at IEEE ISBI 2023, 5 pages

Context-Informed Machine Translation of Manga using Multimodal Large Language Models

Philip Lippmann, Konrad Skublicki, Joshua Tanner, Shonosuke Ishiwatari, Jie Yang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

Comments COLING 2025

SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation

Jonathan Roberts, Kai Han, Neil Houlsby, Samuel Albanie

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at NeurIPS 2024 (Datasets and Benchmarks Track)

Agri-LLaVA: Knowledge-Infused Large Multimodal Assistant on Agricultural Pests and Diseases

Liqiong Wang, Teng Jin, Jinyu Yang, Ales Leonardis, Fangyi Wang, Feng Zheng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

CADMR: Cross-Attention and Disentangled Learning for Multimodal Recommender Systems

Yasser Khalafaoui, Martino Lovisetto, Basarab Matei, Nistor Grozavu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

PKRD-CoT: A Unified Chain-of-thought Prompting for Multi-Modal Large Language Models in Autonomous Driving

Xuewen Luo, Fan Ding, Yinsheng Song, Xiaofeng Zhang, Junnyong Loo

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.AI

Comments This paper has been accepted for presentation at ICONIP 2024

Grid-augmented vision: A simple yet effective approach for enhanced spatial understanding in multi-modal agents

Joongwon Chae, Zhenyu Wang, Lian Zhang, Dongmei Yu, Peiwu Qin

专题命中 多模态评测 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

Comments 14 pages, 11 figures

Improving Multimodal LLMs Ability In Geometry Problem Solving, Reasoning, And Multistep Scoring

Avinash Anand, Raj Jaiswal, Abhishek Dharmadhikari, Atharva Marathe, Harsh Parimal Popat, Harshil Mital, Kritarth Prasad, Rajiv Ratn Shah, Roger Zimmermann

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Comments 15 pages

Fairness at Every Intersection: Uncovering and Mitigating Intersectional Biases in Multimodal Clinical Predictions

Resmi Ramachandranpillai, Kishore Sampath, Ayaazuddin Mohammad, Malihe Alikhani

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Games

Nicholas R. Waytowich, Devin White, MD Sunbeam, Vinicius G. Goecks

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Comments Currently under review

ThermoNeRF: Joint RGB and Thermal Novel View Synthesis for Building Facades using Multimodal Neural Radiance Fields

Mariam Hassan, Florent Forest, Olga Fink, Malcolm Mielle

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

NEMO: Can Multimodal LLMs Identify Attribute-Modified Objects?

Jiaxuan Li, Junwen Mo, MinhDuc Vo, Akihiro Sugimoto, Hideki Nakayama

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV