多模态大模型 - arXivDaily 专题

2304.02313 2024-03-05 cs.CL 79%

Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines

Yaochen Zhu, Xiangqing Shen, Rui Xia

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

2403.01116 2024-03-05 cs.CL 79%

MulCogBench: A Multi-modal Cognitive Benchmark Dataset for Evaluating Chinese and English Computational Language Models

Yunhao Zhang, Xiaohan Zhang, Chong Li, Shaonan Wang, Chengqing Zong

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CL

2402.13693 2024-03-04 cs.CL 79%

CMNER: A Chinese Multimodal NER Dataset based on Social Media

Yuanze Ji, Bobo Li, Jun Zhou, Fei Li, Chong Teng, Donghong Ji

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

2402.16110 2024-02-27 cs.IR cs.MM 79%

Disentangled Graph Variational Auto-Encoder for Multimodal Recommendation with Interpretability

Xin Zhou, Chunyan Miao

专题命中多模态评测：multimodal(title,abstract)；分类 cs.MM

Comments 12 pages, 7 figures

2401.01646 2024-02-27 cs.CV 79%

Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Prediction

Yilan Zhang, Yingxue Xu, Jianqi Chen, Fengying Xie, Hao Chen

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

2310.10221 2024-02-26 cs.RO cs.CV 79%

RoboLLM: Robotic Vision Tasks Grounded on Multimodal Large Language Models

Zijun Long, George Killick, Richard McCreadie, Gerardo Aragon Camarasa

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

2402.14245 2024-02-23 cs.RO cs.AI cs.LG 79%

Enhancing Robotic Manipulation with AI Feedback from Multimodal Large Language Models

Jinyi Liu, Yifu Yuan, Jianye Hao, Fei Ni, Lingzhi Fu, Yibin Chen, Yan Zheng

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI

Comments Presented at AAAI 2024 RL+LLMs Workshop

2402.13232 2024-02-21 cs.CV cs.RO 79%

A Touch, Vision, and Language Dataset for Multimodal Alignment

Letian Fu, Gaurav Datta, Huang Huang, William Chung-Ho Panitch, Jaimyn Drake, Joseph Ortiz, Mustafa Mukadam, Mike Lambeta, Roberto Calandra, Ken Goldberg

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

2305.14836 2024-02-21 cs.CV 79%

NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario

Tianwen Qian, Jingjing Chen, Linhai Zhuo, Yang Jiao, Yu-Gang Jiang

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted to AAAI 2024

2402.12183 2024-02-20 cs.AI 79%

MultiFIX: An XAI-friendly feature inducing approach to building models from multimodal data

Mafalda Malafaia, Thalea Schlender, Peter A. N. Bosman, Tanja Alderliesten

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI

Comments 8 pages, 9 figures

2402.10454 2024-02-19 cs.CV 79%

Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary Task Integration

Mahapara Khurshid, Mayank Vatsa, Richa Singh

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

2402.09107 2024-02-15 cs.CV 79%

Headset: Human emotion awareness under partial occlusions multimodal dataset

Fatemeh Ghorbani Lohesara, Davi Rabbouni Freitas, Christine Guillemot, Karen Eguiazarian, Sebastian Knorr

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

Comments Accepted in ISMAR 2023 and published in IEEE Transactions on Visualization and Computer Graphics Dataset: https://webpages.tuni.fi/headset

2402.01785 2024-02-06 cs.LG cs.AI econ.EM stat.ME stat.ML 79%

DoubleMLDeep: Estimation of Causal Effects with Multimodal Data

Sven Klaassen, Jan Teichert-Kluge, Philipp Bach, Victor Chernozhukov, Martin Spindler, Suhas Vijaykumar

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI

2401.15050 2024-01-29 cs.CL 79%

LongFin: A Multimodal Document Understanding Model for Long Financial Domain Documents

Ahmed Masry, Amir Hajian

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at AAAI 2024 Workshop on AI in Finance for Social Impact

2401.14743 2024-01-29 cs.AI 79%

Synthetic Multimodal Dataset for Empowering Safety and Well-being in Home Environments

Takanori Ugai, Shusaku Egami, Swe Nwe Nwe Htun, Kouji Kozaki, Takahiro Kawamura, Ken Fukuda

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI

Comments 7 pages, 2 figures,4 tables

2401.12074 2024-01-24 eess.IV cs.CV q-bio.NC 79%

DeepCERES: A Deep learning method for cerebellar lobule segmentation using ultra-high resolution multimodal MRI

Sergio Morell-Ortega, Marina Ruiz-Perez, Marien Gadea, Roberto Vivo-Hernando, Gregorio Rubio, Fernando Aparici, Maria de la Iglesia-Vaya, Gwenaelle Catheline, Pierrick Coupé, José V. Manjón

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

Comments 20 pages

2401.09899 2024-01-19 cs.CL 79%

Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations

Prince Jha, Krishanu Maity, Raghav Jain, Apoorv Verma, Sriparna Saha, Pushpak Bhattacharyya

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

Comments EACL2024

2401.04471 2024-01-10 cs.CL 79%

TransportationGames: Benchmarking Transportation Knowledge of (Multimodal) Large Language Models

Xue Zhang, Xiangyu Shi, Xinyue Lou, Rui Qi, Yufeng Chen, Jinan Xu, Wenjuan Han

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

Comments Work in Progress

2401.01955 2024-01-05 cs.HC cs.MM 79%

MULTI-CASE: A Transformer-based Ethics-aware Multimodal Investigative Intelligence Framework

Maximilian T. Fischer, Yannick Metz, Lucas Joos, Matthias Miller, Daniel A. Keim

专题命中多模态评测：multimodal(title,abstract)；分类 cs.MM

Comments 6 pages, 3 figures, 1 table

2204.13750 2024-01-05 cs.NE cs.AI cs.DS math.OC 79%

A First Runtime Analysis of the NSGA-II on a Multimodal Problem

Benjamin Doerr, Zhongdi Qu

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI

Comments Appeared in the Transactions on Evolutionary Computation. Extends a paper that appeared in the Proceedings of PPSN 2022

Journal ref IEEE Transactions on Evolutionary Computation 27(5): 1288-1297 (2023)

2401.01553 2024-01-04 eess.IV cs.CV 79%

Multi-modal Learning with Missing Modality in Predicting Axillary Lymph Node Metastasis

Shichuan Zhang, Sunyi Zheng, Zhongyi Shui, Honglin Li, Lin Yang

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV

2401.00908 2024-01-03 cs.CL 79%

DocLLM: A layout-aware generative language model for multimodal document understanding

Dongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma, Petr Babkin, Simerjot Kaur, Yulong Pei, Armineh Nourbakhsh, Xiaomo Liu

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

Comments 16 pages, 4 figures

2401.00127 2024-01-02 cs.CV cs.SI 79%

Pushing Boundaries: Exploring Zero Shot Object Classification with Large Multimodal Models

Ashhadul Islam, Md. Rafiul Biswas, Wajdi Zaghouani, Samir Brahim Belhaouari, Zubair Shah

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

Comments 5 pages,6 figures, 4 tables, Accepted on The International Symposium on Foundation and Large Language Models (FLLM2023)

Journal ref https://fllm-conference.org/2023/

2312.17624 2024-01-01 cs.LG cs.AI 79%

XAI for In-hospital Mortality Prediction via Multimodal ICU Data

Xingqiao Li, Jindong Gu, Zhiyong Wang, Yancheng Yuan, Bo Du, Fengxiang He

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI

2310.09596 2023-12-27 cs.CL 79%

RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification

Junjie Ye, Jie Zhou, Junfeng Tian, Rui Wang, Qi Zhang, Tao Gui, Xuanjing Huang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

2310.01438 2023-12-25 cs.LG cs.AI 79%

Building Flexible, Scalable, and Machine Learning-ready Multimodal Oncology Datasets

Aakash Tripathi, Asim Waqas, Kavya Venkatesan, Yasin Yilmaz, Ghulam Rasool

专题命中多模态评测：multimodal(title,abstract)；分类 cs.AI

2305.03453 2023-12-19 cs.CL 79%

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answering

Lei Wang, Yi Hu, Jiabang He, Xing Xu, Ning Liu, Hui Liu, Heng Tao Shen

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL

Comments AAAI 2024

2312.09753 2023-12-18 cs.MM 79%

MORE: A Multimodal Object-Entity Relation Extraction Dataset with a Benchmark Evaluation

Liang He, Hongke Wang, Yongchang Cao, Zhen Wu, Jianbing Zhang, Xinyu Dai

专题命中多模态评测：multimodal(title,abstract)；分类 cs.MM

Journal ref In Proceedings of the 31st ACM International Conference on Multimedia, pp. 4564-4573. 2023

2312.07627 2023-12-14 cs.CV cs.LG cs.SI 79%

Multimodal Sentiment Analysis: Perceived vs Induced Sentiments

Aditi Aggarwal, Deepika Varshney, Saurabh Patel

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV

2303.09797 2023-12-14 cs.CV 79%

MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation

Haozhe Wu, Jia Jia, Junliang Xing, Hongwei Xu, Xiangyuan Wang, Jelo Wang

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV

Comments 10 pages, 8 figures. This paper has been submitted to IEEE Transaction on MultiMedia, which is the extension of our MM2023 paper arXiv:2308.05428. The dataset is now publicly available, see Project page at https://wuhaozhe.github.io/mmface4d/

1. 多模态评测 9189 篇

Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines

MulCogBench: A Multi-modal Cognitive Benchmark Dataset for Evaluating Chinese and English Computational Language Models

CMNER: A Chinese Multimodal NER Dataset based on Social Media

Disentangled Graph Variational Auto-Encoder for Multimodal Recommendation with Interpretability

Prototypical Information Bottlenecking and Disentangling for Multimodal Cancer Survival Prediction

RoboLLM: Robotic Vision Tasks Grounded on Multimodal Large Language Models

Enhancing Robotic Manipulation with AI Feedback from Multimodal Large Language Models

A Touch, Vision, and Language Dataset for Multimodal Alignment

NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario

MultiFIX: An XAI-friendly feature inducing approach to building models from multimodal data

Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary Task Integration

Headset: Human emotion awareness under partial occlusions multimodal dataset

DoubleMLDeep: Estimation of Causal Effects with Multimodal Data

LongFin: A Multimodal Document Understanding Model for Long Financial Domain Documents

Synthetic Multimodal Dataset for Empowering Safety and Well-being in Home Environments

DeepCERES: A Deep learning method for cerebellar lobule segmentation using ultra-high resolution multimodal MRI

Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations

TransportationGames: Benchmarking Transportation Knowledge of (Multimodal) Large Language Models

MULTI-CASE: A Transformer-based Ethics-aware Multimodal Investigative Intelligence Framework

A First Runtime Analysis of the NSGA-II on a Multimodal Problem

Multi-modal Learning with Missing Modality in Predicting Axillary Lymph Node Metastasis

DocLLM: A layout-aware generative language model for multimodal document understanding

Pushing Boundaries: Exploring Zero Shot Object Classification with Large Multimodal Models

XAI for In-hospital Mortality Prediction via Multimodal ICU Data

RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification

Building Flexible, Scalable, and Machine Learning-ready Multimodal Oncology Datasets

T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answering

MORE: A Multimodal Object-Entity Relation Extraction Dataset with a Benchmark Evaluation

Multimodal Sentiment Analysis: Perceived vs Induced Sentiments

MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation