多模态大模型

Stark: Social Long-Term Multi-Modal Conversation with Persona Commonsense Knowledge

Young-Jun Lee, Dokyong Lee, Junyoung Youn, Kyeongjin Oh, Byungsoo Ko, Jonghwan Hyeon, Ho-Jin Choi

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Project website: https://stark-dataset.github.io

Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Dataset

Rui Liu, Haolin Zuo, Zheng Lian, Xiaofen Xing, Björn W. Schuller, Haizhou Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 26 pages, 8 figures, 12 tables, NeurIPS 2024 Dataset and Benchmark Track

MARS: Multimodal Active Robotic Sensing for Articulated Characterization

Hongliang Zeng, Ping Zhang, Chengjiong Wu, Jiahua Wang, Tingyu Ye, Fang Li

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

FlowVQA: Mapping Multimodal Logic in Visual Question Answering with Flowcharts

Shubhankar Singh, Purvi Chaurasia, Yerram Varun, Pranshu Pandya, Vatsal Gupta, Vivek Gupta, Dan Roth

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted in ACL 2024 (Findings), 21 pages, 7 figures, 9 Tables

MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations

Hanlei Zhang, Xin Wang, Hua Xu, Qianrui Zhou, Kai Gao, Jianhua Su, jinyue Zhao, Wenrui Li, Yanting Chen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted by ICLR 2024, Long Paper; The abstract is slightly modified due to the length limitation

EHR-Based Mobile and Web Platform for Chronic Disease Risk Prediction Using Large Language Multimodal Models

Chun-Chieh Liao, Wei-Ting Kuo, I-Hsuan Hu, Yen-Chen Shih, Jun-En Ding, Feng Liu, Fang-Ming Hung

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Unveiling the Impact of Multi-Modal Interactions on User Engagement: A Comprehensive Evaluation in AI-driven Conversations

Lichao Zhang, Jia Yu, Shuai Zhang, Long Li, Yangyang Zhong, Guanbao Liang, Yuming Yan, Qing Ma, Fangsheng Weng, Fayu Pan, Jing Li, Renjun Xu, Zhenzhong Lan

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking

Siyuan Wang, Zhuohan Long, Zhihao Fan, Zhongyu Wei

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Plug-and-Play Grounding of Reasoning in Multimodal Large Language Models

Jiaxing Chen, Yuxuan Liu, Dehu Li, Xiang An, Weimo Deng, Ziyong Feng, Yongle Zhao, Yin Xie

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 15 pages, 8 figures

Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding

Tuo Zhang, Tiantian Feng, Yibin Ni, Mengqin Cao, Ruying Liu, Katharine Butler, Yanjun Weng, Mi Zhang, Shrikanth S. Narayanan, Salman Avestimehr

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Uni-SMART: Universal Science Multimodal Analysis and Research Transformer

Hengxing Cai, Xiaochen Cai, Shuwen Yang, Jiankun Wang, Lin Yao, Zhifeng Gao, Junhan Chang, Sihang Li, Mingjun Xu, Changxin Wang, Hongshuai Wang, Yongge Li, Mujie Lin, Yaqi Li, Yuqi Yin, Linfeng Zhang, Guolin Ke

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models

Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu, Muhammad Abdul-Mageed

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

cPAPERS: A Dataset of Situated and Multimodal Interactive Conversations in Scientific Papers

Anirudh Sundar, Jin Xu, William Gay, Christopher Richardson, Larry Heck

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 1 figure

FlexLoc: Conditional Neural Networks for Zero-Shot Sensor Perspective Invariance in Object Localization with Distributed Multimodal Sensors

Jason Wu, Ziqi Wang, Xiaomin Ouyang, Ho Lyun Jeong, Colin Samplawski, Lance Kaplan, Benjamin Marlin, Mani Srivastava

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

BloomVQA: Assessing Hierarchical Multi-modal Comprehension

Yunye Gong, Robik Shrestha, Jared Claypoole, Michael Cogswell, Arijit Ray, Christopher Kanan, Ajay Divakaran

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by ACL Findings (2024). Dataset available at https://huggingface.co/datasets/ygong/BloomVQA

Integrating Text and Image Pre-training for Multi-modal Algorithmic Reasoning

Zijian Zhang, Wei Liu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Evaluation of data inconsistency for multi-modal sentiment analysis

Yufei Wang, Mengyue Wu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

CODIS: Benchmarking Context-Dependent Visual Comprehension for Multimodal Large Language Models

Fuwen Luo, Chi Chen, Zihao Wan, Zhaolu Kang, Qidong Yan, Yingjie Li, Xiaolong Wang, Siyu Wang, Ziyue Wang, Xiaoyue Mi, Peng Li, Ning Ma, Maosong Sun, Yang Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models

Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, Qi Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Project page: https://mm-arxiv.github.io, Camera Ready Version of ACL 2024

Lumos : Empowering Multimodal LLMs with Scene Text Recognition

Ashish Shenoy, Yichao Lu, Srihari Jayakumar, Debojeet Chatterjee, Mohsen Moslehpour, Pierce Chuang, Abhay Harpale, Vikas Bhardwaj, Di Xu, Shicong Zhao, Longfang Zhao, Ankit Ramchandani, Xin Luna Dong, Anuj Kumar

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted to KDD 2024 (ADS Track)

MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception

Yuhao Wang, Yusheng Liao, Heyang Liu, Hongcheng Liu, Yu Wang, Yanfeng Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models

Kai Wu, Boyuan Jiang, Zhengkai Jiang, Qingdong He, Donghao Luo, Shengzhi Wang, Qingwen Liu, Chengjie Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 5 figures with supplementary material

OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 51 pages, 21 figures

Peacock: A Family of Arabic Multimodal Large Language Models and Benchmarks

Fakhraddin Alwajih, El Moatez Billah Nagoudi, Gagan Bhatia, Abdelrahman Mohamed, Muhammad Abdul-Mageed

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving

Jiaxin Zhang, Zhongzhi Li, Mingliang Zhang, Fei Yin, Chenglin Liu, Yashar Moshfeghi

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted in ACL 2024 Findings

PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset

Yang Hou, Haitao Fu, Chuankai Chen, Zida Li, Haoyu Zhang, Jianjun Zhao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI、eess.AS

Comments 13 page, 4 figures

Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots

Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang, Zhixuan Liang, Zeyu Lu, Ying Shan, Ping Luo

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Exploring the Capabilities of Large Multimodal Models on Dense Text

Shuo Zhang, Biao Yang, Zhang Li, Zhiyin Ma, Yuliang Liu, Xiang Bai

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.AI

Multi-Prompt with Depth Partitioned Cross-Modal Learning

Yingjie Tian, Yiqi Wang, Xianda Guo, Zheng Zhu, Long Chen

专题命中 多模态评测 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset

Xinyu Ma, Xuebo Liu, Derek F. Wong, Jun Rao, Bei Li, Liang Ding, Lidia S. Chao, Dacheng Tao, Min Zhang

专题命中 多模态评测 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI