多模态大模型

V$^2$-SfMLearner: Learning Monocular Depth and Ego-motion for Multimodal Wireless Capsule Endoscopy

Long Bai, Beilei Cui, Liangyu Wang, Yanheng Li, Shilong Yao, Sishen Yuan, Yanan Wu, Yang Zhang, Max Q. -H. Meng, Zhen Li, Weiping Ding, Hongliang Ren

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments To appear in IEEE Transactions on Automation Science and Engineering (IEEE TASE)

MineAgent: Towards Remote-Sensing Mineral Exploration with Multimodal Large Language Models

Beibei Yu, Tao Shen, Hongbin Na, Ling Chen, Denqi Li

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Unlocking Cross-Lingual Sentiment Analysis through Emoji Interpretation: A Multimodal Generative AI Approach

Rafid Ishrak Jahan, Heng Fan, Haihua Chen, Yunhe Feng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

AgriBench: A Hierarchical Agriculture Benchmark for Multimodal Large Language Models

Yutong Zhou, Masahiro Ryo

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPPA @ECCV2024. Dataset: https://github.com/Yutong-Zhou-cv/AgriBench

Each Fake News is Fake in its Own Way: An Attribution Multi-Granularity Benchmark for Multimodal Fake News Detection

Hao Guo, Zihan Ma, Zhi Zeng, Minnan Luo, Weixin Zeng, Jiuyang Tang, Xiang Zhao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

STICKERCONV: Generating Multimodal Empathetic Responses from Scratch

Yiqun Zhang, Fanheng Kong, Peidong Wang, Shuang Sun, Lingshuai Wang, Shi Feng, Daling Wang, Yifei Zhang, Kaisong Song

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7707-7733, Bangkok, Thailand, August 2024

Predicting Internet Connectivity in Schools: A Feasibility Study Leveraging Multi-modal Data and Location Encoders in Low-Resource Settings

Kelsey Doerksen, Casper Fibaek, Rochelle Schneider, Do-Hyung Kim, Isabelle Tingzon

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Tangram: Benchmark for Evaluating Geometric Element Recognition in Large Multimodal Models

Chao Zhang, Jiamin Tang, Jing Xiao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 8 figures

ACE-$M^3$: Automatic Capability Evaluator for Multimodal Medical Models

Xiechi Zhang, Shunfan Zheng, Linlin Wang, Gerard de Melo, Zhu Cao, Xiaoling Wang, Liang He

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Seeing the Forest and the Trees: Solving Visual Graph and Tree Based Data Structure Problems using Large Multimodal Models

Sebastian Gutierrez, Irene Hou, Jihye Lee, Kenneth Angelikas, Owen Man, Sophia Mettille, James Prather, Paul Denny, Stephen MacNeil

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 14 pages, 4 figures, to be published in ACE 2025

RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Models

Pengkun Jiao, Xinlan Wu, Bin Zhu, Jingjing Chen, Chong-Wah Ngo, Yugang Jiang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models

Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xucheng Yin, Cheng-lin Liu, Lianwen Jin, Xiang Bai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Piecing It All Together: Verifying Multi-Hop Multimodal Claims

Haoran Wang, Aman Rangapur, Xiongxiao Xu, Yueqing Liang, Haroon Gharwi, Carl Yang, Kai Shu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments COLING 2025

A Dual-Module Denoising Approach with Curriculum Learning for Enhancing Multimodal Aspect-Based Sentiment Analysis

Nguyen Van Doan, Dat Tran Nguyen, Cam-Van Thi Nguyen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted at PACLIC 2024

Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models

Sherzod Hakimov, Yerkezhan Abdullayeva, Kushal Koshti, Antonia Schmidt, Yan Weiser, Anne Beyer, David Schlangen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted at COLING 2025

Biological Brain Age Estimation using Sex-Aware Adversarial Variational Autoencoder with Multimodal Neuroimages

Abd Ur Rehman, Azka Rehman, Muhammad Usman, Abdullah Shahid, Sung-Min Gho, Aleum Lee, Tariq M. Khan, Imran Razzak

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Visual Error Patterns in Multi-Modal AI: A Statistical Approach

Ching-Yi Wang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

MMAUD: A Comprehensive Multi-Modal Anti-UAV Dataset for Modern Miniature Drone Threats

Shenghai Yuan, Yizhuo Yang, Thien Hoang Nguyen, Thien-Minh Nguyen, Jianfei Yang, Fen Liu, Jianping Li, Han Wang, Lihua Xie

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICRA 2024

VisScience: An Extensive Benchmark for Evaluating K12 Educational Multi-modal Scientific Reasoning

Zhihuan Jiang, Zhen Yang, Jinhao Chen, Zhengxiao Du, Weihan Wang, Bin Xu, Jie Tang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 89 pages, 70 figures

MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model

Zhen Yang, Jinhao Chen, Zhengxiao Du, Wenmeng Yu, Weihan Wang, Wenyi Hong, Zhihuan Jiang, Bin Xu, Jie Tang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 30 pages,19 figures

Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey

Jiayi Kuang, Jingyou Xie, Haohao Luo, Ronghao Li, Zhe Xu, Xianfeng Cheng, Yinghui Li, Xika Lin, Ying Shen

专题命中 多模态评测 ：MLLM(title)；multimodal(abstract)；分类 cs.CV、cs.CL

Exploring Large Language Models for Multimodal Sentiment Analysis: Challenges, Benchmarks, and Future Directions

Shezheng Song

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Multimodal large language model for wheat breeding: a new exploration of smart breeding

Guofeng Yang, Yu Li, Yong He, Zhenjiang Zhou, Lingzhen Ye, Hui Fang, Yiqi Luo, Xuping Feng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, Chaowei Xiao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Localizing Events in Videos with Multimodal Queries

Gengyuan Zhang, Mang Ling Ada Fok, Jialu Ma, Yan Xia, Daniel Cremers, Philip Torr, Volker Tresp, Jindong Gu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 20 pages (including references and appendix); for the project homepage, see https://icq-benchmark.github.io/

MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis

Yingjie Zhou, Zicheng Zhang, Jiezhang Cao, Jun Jia, Yanwei Jiang, Farong Wen, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Multi-Modal Self-Supervised Learning for Surgical Feedback Effectiveness Assessment

Arushi Gupta, Rafal Kocielnik, Jiayun Wang, Firdavs Nasriddinov, Cherine Yang, Elyssa Wong, Anima Anandkumar, Andrew Hung

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted as a spotlight proceedings paper at Machine Learning for Health 2024

Multi-modal Situated Reasoning in 3D Scenes

Xiongkun Linghu, Jiangyong Huang, Xuesong Niu, Xiaojian Ma, Baoxiong Jia, Siyuan Huang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by NeurIPS 2024 Datasets and Benchmarks Track. Project page: https://msr3d.github.io/

Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment

Danqing Ma, Meng Wang, Ao Xiang, Zongqing Qi, Qin Yang

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Multi-Task Adversarial Variational Autoencoder for Estimating Biological Brain Age with Multimodal Neuroimaging

Muhammad Usman, Azka Rehman, Abdullah Shahid, Abd Ur Rehman, Sung-Min Gho, Aleum Lee, Tariq M. Khan, Imran Razzak

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI