多模态大模型 - arXivDaily 专题

2410.04780 2025-02-19 cs.CV 83%

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, Xuming Hu

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted by The Thirteenth International Conference on Learning Representations (ICLR 2025)

2502.05641 2025-02-11 cs.RO cs.AI 83%

Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs

Aayam Shrestha, Pan Liu, German Ros, Kai Yuan, Alan Fern

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI

Journal ref The European Conference on Computer Vision (ECCV), 2024

2502.03999 2025-02-07 eess.IV cs.CV 83%

A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma

Ahmed Gomaa, Yixing Huang, Pluvio Stephan, Katharina Breininger, Benjamin Frey, Arnd Dörfler, Oliver Schnell, Daniel Delev, Roland Coras, Charlotte Schmitter, Jenny Stritzelberger, Sabine Semrau, Andreas Maier, Siming Bayer, Stephan Schönecker, Dieter H Heiland, Peter Hau, Udo S. Gaipl, Christoph Bert, Rainer Fietkau, Manuel A. Schmidt, Florian Putz

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2501.15795 2025-01-28 cs.CV 83%

Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?

Zhiling Chen, Hanning Chen, Mohsen Imani, Farhad Imani

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments 16 pages, 11 figures

2501.11535 2025-01-22 cs.CV 83%

A baseline for machine-learning-based hepatocellular carcinoma diagnosis using multi-modal clinical data

Binwu Wang, Isaac Rodriguez, Leon Breitinger, Fabian Tollens, Timo Itzel, Dennis Grimm, Andrei Sirazitdinov, Matthias Frölich, Stefan Schönberg, Andreas Teufel, Jürgen Hesser, Wenzhao Zhao

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2501.07819 2025-01-15 cs.CV 83%

3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding

Haomiao Xiong, Yunzhi Zhuge, Jiawen Zhu, Lu Zhang, Huchuan Lu

专题命中多模态评测：multimodal(title)；multi-modal(abstract)；MLLM(abstract)；分类 cs.CV

Comments Accepted to IEEE Transactions on Multimedia (TMM)

2501.05444 2025-01-10 cs.CV 83%

Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark

Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, Yu Cheng

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

2412.14056 2024-12-19 cs.CV cs.AI cs.CL cs.LG cs.MM 83%

A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future

Shilin Sun, Wenbin An, Feng Tian, Fang Nan, Qidong Liu, Jun Liu, Nazaraf Shah, Ping Chen

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments This work has been submitted to the IEEE for possible publication

2412.10594 2024-12-17 cs.CV cs.LG 83%

Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics

Sara Ghazanfari, Siddharth Garg, Nicolas Flammarion, Prashanth Krishnamurthy, Farshad Khorrami, Francesco Croce

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2410.21108 2024-12-11 cs.CV 83%

LiGAR: LiDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition

Naga Venkata Sai Raviteja Chappa, Khoa Luu

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted at WACV 2025; 14 pages, 4 figures, 10 tables

2412.00955 2024-12-05 cs.CV 83%

WAFFLE: Multimodal Floorplan Understanding in the Wild

Keren Ganon, Morris Alper, Rachel Mikulinsky, Hadar Averbuch-Elor

专题命中多模态评测：multimodal(title,abstract)；multimodal foundation model(abstract)；分类 cs.CV

Comments Accepted to WACV 2025. Project page: https://tau-vailab.github.io/WAFFLE/

2412.01249 2024-12-03 cs.CL 83%

Data Uncertainty-Aware Learning for Multimodal Aspect-based Sentiment Analysis

Hao Yang, Zhenyu Zhang, Yanyan Zhao, Bing Qin

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL

2407.13596 2024-12-02 cs.CV 83%

EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing

Wei Zhang, Miaoxin Cai, Tong Zhang, Jun Li, Yin Zhuang, Xuerui Mao

专题命中多模态评测：multi-modal(title,abstract)；MLLM(abstract)；分类 cs.CV

2411.06735 2024-11-22 cs.AI 83%

Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data

Kai Kim, Howard Tsai, Rajat Sen, Abhimanyu Das, Zihao Zhou, Abhishek Tanpure, Mathew Luo, Rose Yu

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.AI

Comments 21 pages, 4 tables, 2 figures

2411.11360 2024-11-19 cs.CV 83%

CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset

Zhiming Wang, Mingze Wang, Sheng Xu, Yanjing Li, Baochang Zhang

专题命中多模态评测：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

2410.24046 2024-11-01 eess.IV cs.CV cs.LG 83%

Deep Learning with HM-VGG: AI Strategies for Multi-modal Image Analysis

Junliang Du, Yiru Cang, Tong Zhou, Jiacheng Hu, Weijie He

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2405.05956 2024-10-29 cs.RO cs.CV 83%

Probing Multimodal LLMs as World Models for Driving

Shiva Sreeram, Tsun-Hsuan Wang, Alaa Maalouf, Guy Rosman, Sertac Karaman, Daniela Rus

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

Comments https://github.com/sreeramsa/DriveSim https://www.youtube.com/watch?v=Fs8jgngOJzU

2209.14272 2024-10-28 cs.LG cs.CL cs.CV cs.MM cs.SD eess.AS 83%

Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

Lukas Christ, Shahin Amiriparian, Alexander Kathan, Niklas Müller, Andreas König, Björn W. Schuller

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

Comments This work has been submitted to the IEEE for possible publication (Major Revision)

2410.13859 2024-10-18 cs.CV 83%

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

Yaxin Luo, Gen Luo, Jiayi Ji, Yiyi Zhou, Xiaoshuai Sun, Zhiqiang Shen, Rongrong Ji

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2410.09962 2024-10-16 cs.CV 83%

LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models

Han Qiu, Jiaxing Huang, Peng Gao, Qin Qi, Xiaoqin Zhang, Ling Shao, Shijian Lu

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CV

2410.11538 2024-10-16 cs.CV 83%

MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark

Bin Shan, Xiang Fei, Wei Shi, An-Lan Wang, Guozhi Tang, Lei Liao, Jingqun Tang, Xiang Bai, Can Huang

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments 12 pages, 5 figures, project page: https://github.com/xfey/MCTBench?tab=readme-ov-file

2401.06153 2024-10-11 cs.NE cs.AI 83%

Multimodal Optimization with k-Cluster Big Bang-Big Crunch Algorithm and Postprocessing Methods for Identification and Quantification of Optima

Kemal Erdem Yenin, Reha Oguz Sayin, Kuzey Arar, Kadir Kaan Atalay, Fabio Stroppa

专题命中多模态评测：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.AI

Comments 18 pages

2409.20559 2024-10-01 cs.LG cs.CV 83%

Supervised Multi-Modal Fission Learning

Lingchao Mao, Qi wang, Yi Su, Fleming Lure, Jing Li

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

2409.15125 2024-09-24 cs.CV 83%

Detect, Describe, Discriminate: Moving Beyond VQA for MLLM Evaluation

Manu Gaur, Darshan Singh S, Makarand Tapaswi

专题命中多模态评测：MLLM(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments ECCV 2024 Workshop EVAL-FoMo; Project Page: https://katha-ai.github.io/projects/detect-describe-discriminate/

2409.13440 2024-09-23 eess.SP cs.AI cs.CR cs.LG 83%

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG Representative Learning

Xiaowen Fu, Bingxin Wang, Xinzhou Guo, Guoqing Liu, Yang Xiang

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

2408.12682 2024-08-26 cs.LG cs.AI cs.CL cs.CV cs.MM 83%

MultiMed: Massively Multimodal and Multitask Medical Understanding

Shentong Mo, Paul Pu Liang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2408.12480 2024-08-26 cs.LG cs.CL 83%

Vintern-1B: An Efficient Multimodal Large Language Model for Vietnamese

Khang T. Doan, Bao G. Huynh, Dung T. Hoang, Thuc D. Pham, Nhat H. Pham, Quan T. M. Nguyen, Bang Q. Vo, Suong N. Hoang

专题命中多模态评测：multimodal(title,abstract)；MLLM(abstract)；分类 cs.CL

2407.10474 2024-07-16 cs.MM 83%

Multi-source Knowledge Enhanced Graph Attention Networks for Multimodal Fact Verification

Han Cao, Lingwei Wei, Wei Zhou, Songlin Hu

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.MM

Comments Accepted by ICME 2024

2407.10125 2024-07-16 cs.CV 83%

When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset

Yi Zhang, Wang Zeng, Sheng Jin, Chen Qian, Ping Luo, Wentao Liu

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Accepted to ECCV'2024

2407.02946 2024-07-04 cs.CV 83%

3D Multimodal Image Registration for Plant Phenotyping

Eric Stumpe, Gernot Bodner, Francesco Flagiello, Matthias Zeppelzauer

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 53 pages, 13 Figures, preprint submitted to Computers and Electronics in Agriculture

1. 多模态评测 9150 篇

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputs

A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma

Can Multimodal Large Language Models be Guided to Improve Industrial Anomaly Detection?

A baseline for machine-learning-based hepatocellular carcinoma diagnosis using multi-modal clinical data

3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding

Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark

A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future

Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics

LiGAR: LiDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition

WAFFLE: Multimodal Floorplan Understanding in the Wild

Data Uncertainty-Aware Learning for Multimodal Aspect-based Sentiment Analysis

EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing

Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data

CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Dataset

Deep Learning with HM-VGG: AI Strategies for Multi-modal Image Analysis

Probing Multimodal LLMs as World Models for Driving

Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

$γ-$MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models

LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models

MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark

Multimodal Optimization with k-Cluster Big Bang-Big Crunch Algorithm and Postprocessing Methods for Identification and Quantification of Optima

Supervised Multi-Modal Fission Learning

Detect, Describe, Discriminate: Moving Beyond VQA for MLLM Evaluation

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG Representative Learning

MultiMed: Massively Multimodal and Multitask Medical Understanding

Vintern-1B: An Efficient Multimodal Large Language Model for Vietnamese

Multi-source Knowledge Enhanced Graph Attention Networks for Multimodal Fact Verification

When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset

3D Multimodal Image Registration for Plant Phenotyping