多模态大模型

MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks

Letitia Parcalabescu, Anette Frank

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published at ACL 2023 Main (Toronto); 10 pages, 14 appendix pages, 11 figures, 3 tables

Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models

Yupan Huang, Zaiqiao Meng, Fangyu Liu, Yixuan Su, Nigel Collier, Yutong Lu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments ICLR 2024 Workshop (Navigating and Addressing Data Problems for Foundation Models)

Visual Language Tracking with Multi-modal Interaction: A Robust Benchmark

Xuchen Li, Shiyu Hu, Xiaokun Feng, Dailing Zhang, Meiqi Wu, Jing Zhang, Kaiqi Huang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Under Review

Nonverbal Immediacy Analysis in Education: A Multimodal Computational Model

Uroš Petković, Jonas Frenkel, Olaf Hellwich, Rebecca Lazarides

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 3 figures. Camera-ready version for the SAB 2024: 17th International Conference on the Simulation of Adaptive Behavior

Towards Patronizing and Condescending Language in Chinese Videos: A Multimodal Dataset and Detector

Hongbo Wang, Junyu Lu, Yan Han, Kai Ma, Liang Yang, Hongfei Lin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Under review in ICASSP 2025

PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis

Meng Luo, Hao Fei, Bobo Li, Shengqiong Wu, Qian Liu, Soujanya Poria, Erik Cambria, Mong-Li Lee, Wynne Hsu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted by ACM MM 2024 (Oral)

FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models

Yixuan Li, Xuelin Liu, Xiaoyang Wang, Bu Sung Lee, Shiqi Wang, Anderson Rocha, Weisi Lin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms

Yiqiao Jin, Minje Choi, Gaurav Verma, Jindong Wang, Srijan Kumar

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments In Proceedings of ACL 2024

Knowledge-Aware Reasoning over Multimodal Semi-structured Tables

Suyash Vardhan Mathur, Jainit Sushil Bafna, Kunal Kartik, Harshita Khandelwal, Manish Shrivastava, Vivek Gupta, Mohit Bansal, Dan Roth

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Exploring Multi-modal Neural Scene Representations With Applications on Thermal Imaging

Mert Özer, Maximilian Weiherer, Martin Hundhausen, Bernhard Egger

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted to ECCVW'24

Target-Dependent Multimodal Sentiment Analysis Via Employing Visual-to Emotional-Caption Translation Network using Visual-Caption Pairs

Ananya Pandey, Dinesh Kumar Vishwakarma

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Characterizing Multimodal Long-form Summarization: A Case Study on Financial Reports

Tianyu Cao, Natraj Raman, Danial Dervovic, Chenhao Tan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

PathInsight: Instruction Tuning of Multimodal Datasets and Models for Intelligence Assisted Diagnosis in Histopathology

Xiaomin Wu, Rui Xu, Pengchen Wei, Wenkang Qin, Peixiang Huang, Ziheng Li, Lin Luo

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 10 pages, 2 figures

User-in-the-loop Evaluation of Multimodal LLMs for Activity Assistance

Mrinal Verghese, Brian Chen, Hamid Eghbalzadeh, Tushar Nagarajan, Ruta Desai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 9 pages, 4 figures

An analysis of HOI: using a training-free method with multimodal visual foundation models when only the test set is available, without the training set

Chaoyi Ai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs

Peng Ding, Jingyu Wu, Jun Kuang, Dan Ma, Xuezhi Cao, Xunliang Cai, Shi Chen, Jiajun Chen, Shujian Huang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments Acccepted by ACM MM 2024, 14 pages, 11 figures, 9 tables

D-Rax: Domain-specific Radiologic assistant leveraging multi-modal data and eXpert model predictions

Hareem Nisar, Syed Muhammad Anwar, Zhifan Jiang, Abhijeet Parida, Ramon Sanchez-Jacob, Vishwesh Nath, Holger R. Roth, Marius George Linguraru

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments accepted to the MICCAI 2024 Second International Workshop on Foundation Models for General Medical AI

ERIT Lightweight Multimodal Dataset for Elderly Emotion Recognition and Multimodal Fusion Evaluation

Rita Frieske, Bertram E. Shi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

GigaPevt: Multimodal Medical Assistant

Pavel Blinov, Konstantin Egorov, Ivan Sviridov, Nikolay Ivanov, Stepan Botman, Evgeniy Tagin, Stepan Kudin, Galina Zubkova, Andrey Savchenko

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments IJCAI 2024, 4 pages, 2 figures, 2 tables

Journal ref Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI) Demo Track, 2024, pp. 8614-8618

Multi-modal Imaging Genomics Transformer: Attentive Integration of Imaging with Genomic Biomarkers for Schizophrenia Classification

Nagur Shareef Shaik, Teja Krishna Cherukuri, Vince D. Calhoun, Dong Hye Ye

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for presentation at the AI for Imaging Genomic Learning (AIIG) Workshop, MICCAI 2024

ConTextual: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models

Rohan Wadhawan, Hritik Bansal, Kai-Wei Chang, Nanyun Peng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref PMLR 235:49733-49787, 2024

Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic

Fakhraddin Alwajih, Gagan Bhatia, Muhammad Abdul-Mageed

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

3MVRD: Multimodal Multi-task Multi-teacher Visually-Rich Form Document Understanding

Yihao Ding, Lorenzo Vaiani, Caren Han, Jean Lee, Paolo Garza, Josiah Poon, Luca Cagliero

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at Findings of ACL 2024

I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition

Yannis Vasilakis, Rachel Bittner, Johan Pauwels

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to ISMIR 2024

How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts

Yusu Qian, Haotian Zhang, Yinfei Yang, Zhe Gan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Bosch Street Dataset: A Multi-Modal Dataset with Imaging Radar for Automated Driving

Karim Armanious, Maurice Quach, Michael Ulrich, Timo Winterling, Johannes Friesen, Sascha Braun, Daniel Jenet, Yuri Feldman, Eitan Kosman, Philipp Rapp, Volker Fischer, Marc Sons, Lukas Kohns, Daniel Eckstein, Daniela Egbert, Simone Letsch, Corinna Voege, Felix Huttner, Alexander Bartler, Robert Maiwald, Yancong Lin, Ulf Rüegg, Claudius Gläser, Bastian Bischoff, Jascha Freess, Karsten Haug, Kathrin Klee, Holger Caesar

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Visualization Literacy of Multimodal Large Language Models: A Comparative Study

Zhimin Li, Haichao Miao, Valerio Pascucci, Shusen Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents

Shrestha Mohanty, Negar Arabzadeh, Andrea Tupini, Yuxuan Sun, Alexey Skrynnik, Artem Zholus, Marc-Alexandre Côté, Julia Kiseleva

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

RoBus: A Multimodal Dataset for Controllable Road Networks and Building Layouts Generation

Tao Li, Ruihang Li, Huangnan Zheng, Shanding Ye, Shijian Li, Zhijie Pan

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

TransMA: an explainable multi-modal deep learning model for predicting properties of ionizable lipid nanoparticles in mRNA delivery

Kun Wu, Zixu Wang, Xiulong Yang, Yangyang Chen, Zhenqi Han, Jialu Zhang, Lizhuang Liu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 9 figures