多模态大模型 - arXivDaily 专题

2405.16473 2024-05-28 cs.CV cs.AI cs.CL 82%

M$^3$CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought

Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, Wanxiang Che

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at ACL2024 Main Conference

2401.14011 2024-05-09 cs.CL cs.AI cs.MM 82%

CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoning

Zheqi He, Xinya Wu, Pengfei Zhou, Richeng Xuan, Guang Liu, Xi Yang, Qiannan Zhu, Hua Huang

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

2405.02287 2024-05-06 cs.CL cs.AI cs.CV 82%

Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models

Piotr Padlewski, Max Bain, Matthew Henderson, Zhongkai Zhu, Nishant Relan, Hai Pham, Donovan Ong, Kaloyan Aleksiev, Aitor Ormazabal, Samuel Phua, Ethan Yeo, Eugenie Lamprecht, Qi Liu, Yuqi Wang, Eric Chen, Deyu Fu, Lei Li, Che Zheng, Cyprien de Masson d'Autume, Dani Yogatama, Mikel Artetxe, Yi Tay

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2404.17238 2024-04-29 cs.IR 82%

TruthSR: Trustworthy Sequential Recommender Systems via User-generated Multimodal Content

Meng Yan, Haibin Huang, Ying Liu, Juan Zhao, Xiyue Gao, Cai Xu, Ziyu Guan, Wei Zhao

专题命中多模态评测：multimodal(title)；multi-modal(abstract)；cross-modal(abstract)

2203.00456 2024-04-17 cs.HC eess.SP 82%

WEMAC: Women and Emotion Multi-modal Affective Computing dataset

Jose A. Miranda, Esther Rituerto-González, Laura Gutiérrez-Martín, Clara Luis-Mingueza, Manuel F. Canabal, Alberto Ramírez Bárcenas, Jose M. Lanza-Gutiérrez, Carmen Peláez-Moreno, Celia López-Ongil

专题命中多模态评测：multi-modal(title,abstract)；audio-visual(abstract)

2403.09451 2024-03-15 cs.CV cs.MM cs.SD eess.AS 82%

M&M: Multimodal-Multitask Model Integrating Audiovisual Cues in Cognitive Load Assessment

Long Nguyen-Phuoc, Renald Gaboriau, Dimitri Delacroix, Laurent Navarro

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.MM、eess.AS

Journal ref Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2 VISAPP: VISAPP, 869-876, 2024 , Rome, Italy

2403.07182 2024-03-13 cs.NE 82%

MAP-Elites with Transverse Assessment for Multimodal Problems in Creative Domains

Marvin Zammit, Antonios Liapis, Georgios N. Yannakakis

专题命中多模态评测：multimodal(title,abstract)；image-text(abstract)

Comments 18 pages, 5 figures To be published in the proceedings of the 13th International Conference on Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART) 2024

2307.00209 2024-03-12 cs.CV cs.AI cs.CL 82%

Image Matters: A New Dataset and Empirical Study for Multimodal Hyperbole Detection

Huixuan Zhang, Xiaojun Wan

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by LREC-COLING 2024

2401.15030 2024-03-05 cs.LG 82%

On the generalization capacity of neural networks during generic multimodal reasoning

Takuya Ito, Soham Dan, Mattia Rigotti, James Kozloski, Murray Campbell

专题命中多模态评测：multimodal(title,abstract)；cross-modal(abstract)

Comments ICLR 2024

2402.16836 2024-02-27 cs.RO cs.AI cs.CL cs.CV 82%

PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models

Dingkun Guo, Yuqi Xiang, Shuqi Zhao, Xinghao Zhu, Masayoshi Tomizuka, Mingyu Ding, Wei Zhan

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2402.15116 2024-02-26 cs.CV cs.AI cs.CL 82%

Large Multimodal Agents: A Survey

Junlin Xie, Zhihong Chen, Ruifei Zhang, Xiang Wan, Guanbin Li

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 15 pages, 4 figures

2402.14804 2024-02-23 cs.CV cs.AI cs.CL cs.LG math.HO 82%

Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset

Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, Hongsheng Li

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2402.04534 2024-02-19 cs.GR 82%

M2fNet: Multi-modal Forest Monitoring Network on Large-scale Virtual Dataset

Yawen Lu, Yunhan Huang, Su Sun, Tansi Zhang, Xuewen Zhang, Songlin Fei, Yingjie Chen

专题命中多模态评测：multi-modal(title,abstract)；multimodal(abstract)

Comments 5 pages

2402.07023 2024-02-13 cs.CL cs.AI cs.CV cs.HC cs.LG 82%

Gemini Goes to Med School: Exploring the Capabilities of Multimodal Large Language Models on Medical Challenge Problems & Hallucinations

Ankit Pal, Malaikannan Sankarasubbu

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Preprint version, Under Review

2402.01730 2024-02-07 cs.CL cs.AI cs.CV 82%

Evaluating LLM -- Generated Multimodal Diagnosis from Medical Images and Symptom Analysis

Dimitrios P. Panagoulias, Maria Virvou, George A. Tsihrintzis

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Department of Informatics, University of Piraeus, Greece

2401.09446 2024-01-19 cs.CV cs.AI cs.CL cs.LG 82%

Explainable Multimodal Sentiment Analysis on Bengali Memes

Kazi Toufique Elahi, Tasnuva Binte Rahman, Shakil Shahriar, Samir Sarker, Sajib Kumar Saha Joy, Faisal Muhammad Shah

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2312.17642 2024-01-01 cs.AI cs.CL cs.CV cs.SI 82%

Research on the Laws of Multimodal Perception and Cognition from a Cross-cultural Perspective -- Taking Overseas Chinese Gardens as an Example

Ran Chen, Xueqi Yao, Jing Zhao, Shuhan Xu, Sirui Zhang, Yijun Mao

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 16 figures,1 table

2109.01537 2023-12-27 cs.CL cs.AI cs.DB cs.MM 82%

A Longitudinal Multi-modal Dataset for Dementia Monitoring and Diagnosis

Dimitris Gkoumas, Bo Wang, Adam Tsakalidis, Maria Wolters, Arkaitz Zubiaga, Matthew Purver, Maria Liakata

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

2308.09936 2023-12-19 cs.CV cs.AI cs.CL cs.LG 82%

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

Wenbo Hu, Yifan Xu, Yi Li, Weiyue Li, Zeyuan Chen, Zhuowen Tu

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at AAAI Conference on Artificial Intelligence (AAAI-24)

2312.08695 2023-12-15 cs.CV cs.AI cs.MM 82%

CPST: Comprehension-Preserving Style Transfer for Multi-Modal Narratives

Yi-Chun Chen, Arnav Jhala

专题命中多模态评测：multi-modal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

2311.17136 2023-11-30 cs.CV cs.AI cs.CL cs.IR 82%

UniIR: Training and Benchmarking Universal Multimodal Information Retrievers

Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, Wenhu Chen

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Our code and dataset are available on this project page: https://tiger-ai-lab.github.io/UniIR/

2311.16941 2023-11-29 cs.LG cs.AI cs.CL cs.CV stat.ME 82%

Debiasing Multimodal Models via Causal Information Minimization

Vaidehi Patil, Adyasha Maharana, Mohit Bansal

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments EMNLP 2023 Findings (16 pages)

2309.01026 2023-10-03 cs.AI cs.CL cs.IR cs.LG cs.MM 82%

Zero-Shot Recommendations with Pre-Trained Large Language Models for Multimodal Nudging

Rachel M. Harrison, Anton Dereventsov, Anton Bibin

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

2309.12460 2023-09-27 cs.LG cs.AI cs.CE cs.CL cs.CV 82%

Multimodal Deep Learning for Scientific Imaging Interpretation

Abdulelah S. Alshehri, Franklin L. Lee, Shihu Wang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2309.03064 2023-09-07 cs.CL cs.AI cs.CV 82%

A Multimodal Analysis of Influencer Content on Twitter

Danae Sánchez Villegas, Catalina Goanta, Nikolaos Aletras

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at AACL 2023

2307.07518 2023-07-18 cs.AI cs.CL cs.CV eess.IV 82%

CephGPT-4: An Interactive Multimodal Cephalometric Measurement and Diagnostic System with Visual Large Language Model

Lei Ma, Jincong Han, Zhaoxin Wang, Dian Zhang

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2305.17678 2023-06-21 cs.CL cs.AI cs.CV 82%

Decoding the Underlying Meaning of Multimodal Hateful Memes

Ming Shan Hee, Wen-Haw Chong, Roy Ka-Wei Lee

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 9 pages. Accepted by IJCAI 2023

2306.00503 2023-06-02 cs.CL cs.AI cs.CV cs.LG 82%

MEWL: Few-shot multimodal word learning with referential uncertainty

Guangyuan Jiang, Manjie Xu, Shiji Xin, Wei Liang, Yujia Peng, Chi Zhang, Yixin Zhu

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at ICML 2023

2302.14680 2023-03-16 cs.CL cs.AI cs.CV 82%

Which One Are You Referring To? Multimodal Object Identification in Situated Dialogue

Holy Lovenia, Samuel Cahyawijaya, Pascale Fung

专题命中多模态评测：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted at EACL SRW 2023

2207.02918 2023-02-16 cs.SE 82%

Heterogeneous Anomaly Detection for Software Systems via Attentive Multi-modal Learning

Baitong Li, Tianyi Yang, Zhuangbin Chen, Yuxin Su, Yongqiang Yang, Michael R. Lyu

专题命中多模态评测：multi-modal(title,abstract)；cross-modal(abstract)

Comments The updated version of this paper has been submitted at arXiv:2302.06914 via another account. This duplicated and old version should be discarded

1. 多模态评测 9150 篇

M$^3$CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought

CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoning

Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models

TruthSR: Trustworthy Sequential Recommender Systems via User-generated Multimodal Content

WEMAC: Women and Emotion Multi-modal Affective Computing dataset

M&M: Multimodal-Multitask Model Integrating Audiovisual Cues in Cognitive Load Assessment

MAP-Elites with Transverse Assessment for Multimodal Problems in Creative Domains

Image Matters: A New Dataset and Empirical Study for Multimodal Hyperbole Detection

On the generalization capacity of neural networks during generic multimodal reasoning

PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models

Large Multimodal Agents: A Survey

Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset

M2fNet: Multi-modal Forest Monitoring Network on Large-scale Virtual Dataset

Gemini Goes to Med School: Exploring the Capabilities of Multimodal Large Language Models on Medical Challenge Problems & Hallucinations

Evaluating LLM -- Generated Multimodal Diagnosis from Medical Images and Symptom Analysis

Explainable Multimodal Sentiment Analysis on Bengali Memes

Research on the Laws of Multimodal Perception and Cognition from a Cross-cultural Perspective -- Taking Overseas Chinese Gardens as an Example

A Longitudinal Multi-modal Dataset for Dementia Monitoring and Diagnosis

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

CPST: Comprehension-Preserving Style Transfer for Multi-Modal Narratives

UniIR: Training and Benchmarking Universal Multimodal Information Retrievers

Debiasing Multimodal Models via Causal Information Minimization

Zero-Shot Recommendations with Pre-Trained Large Language Models for Multimodal Nudging

Multimodal Deep Learning for Scientific Imaging Interpretation

A Multimodal Analysis of Influencer Content on Twitter

CephGPT-4: An Interactive Multimodal Cephalometric Measurement and Diagnostic System with Visual Large Language Model

Decoding the Underlying Meaning of Multimodal Hateful Memes

MEWL: Few-shot multimodal word learning with referential uncertainty

Which One Are You Referring To? Multimodal Object Identification in Situated Dialogue

Heterogeneous Anomaly Detection for Software Systems via Attentive Multi-modal Learning