多模态大模型

Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generation

Aman Rangapur, Haoran Wang, Ling Jian, Kai Shu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Comments 8 pages, 4 figures, 4 tables

Multimodal Information Interaction for Medical Image Segmentation

Xinxin Fan, Lin Liu, Haoran Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Leveraging Large Language Models for Multimodal Search

Oriol Barbany, Michael Huang, Xinliang Zhu, Arnab Dhua

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Published at CVPRW 2024

DesignProbe: A Graphic Design Benchmark for Multimodal Large Language Models

Jieru Lin, Danqing Huang, Tiejun Zhao, Dechen Zhan, Chin-Yew Lin

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments work in progress

RoboFusion: Towards Robust Multi-Modal 3D Object Detection via SAM

Ziying Song, Guoxing Zhang, Lin Liu, Lei Yang, Shaoqing Xu, Caiyan Jia, Feiyang Jia, Li Wang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Sequential Compositional Generalization in Multimodal Models

Semih Yagcioglu, Osman Batur İnce, Aykut Erdem, Erkut Erdem, Desmond Elliott, Deniz Yuret

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to the main conference of NAACL (2024) as a long paper

XoFTR: Cross-modal Feature Matching Transformer

Önder Tuzcuoğlu, Aybora Köksal, Buğra Sofu, Sinan Kalkan, A. Aydın Alatan

专题命中 多模态评测 ：cross-modal(title,abstract)；分类 cs.CV

Comments CVPR Image Matching Workshop, 2024. 12 pages, 7 figures, 5 tables. Codes and dataset are available at https://github.com/OnderT/XoFTR

RLEMMO: Evolutionary Multimodal Optimization Assisted By Deep Reinforcement Learning

Hongqiao Lian, Zeyuan Ma, Hongshu Guo, Ting Huang, Yue-Jiao Gong

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted as full paper at GECCO 2024

Post-hurricane building damage assessment using street-view imagery and structured data: A multi-modal deep learning approach

Zhuoqun Xue, Xiaojian Zhang, David O. Prevatt, Jennifer Bridge, Susu Xu, Xilei Zhao

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

CityNet: A Comprehensive Multi-Modal Urban Dataset for Advanced Research in Urban Computing

Zhengfei Zheng, Xu Geng, Hai Yang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.AI

Unified Multi-modal Diagnostic Framework with Reconstruction Pre-training and Heterogeneity-combat Tuning

Yupei Zhang, Li Pan, Qiushi Yang, Tan Li, Zhen Chen

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Comments to be published in IEEE JBHI; Code available at https://github.com/helenypzhang/UMD

360+x: A Panoptic Multi-modal Scene Understanding Dataset

Hao Chen, Yuqi Hou, Chenyuan Qu, Irene Testini, Xiaohan Hong, Jianbo Jiao

专题命中 多模态评测 ：multi-modal(title)；分类 cs.CV、cs.AI、cs.MM

Comments CVPR 2024 (Oral Presentation), Project page: https://x360dataset.github.io/

Journal ref The IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) 2024

Interpretable 3D Multi-Modal Residual Convolutional Neural Network for Mild Traumatic Brain Injury Diagnosis

Hanem Ellethy, Viktor Vegh, Shekhar S. Chandra

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by the Australasian Joint Conference on Artificial Intelligence 2023 (AJCAI 2023). 12 pages and 5 Figures

MMVP: A Multimodal MoCap Dataset with Vision and Pressure Sensors

He Zhang, Shenghao Ren, Haolei Yuan, Jianhui Zhao, Fan Li, Shuangpeng Sun, Zhenghao Liang, Tao Yu, Qiu Shen, Xun Cao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments CVPR2024

MTMMC: A Large-Scale Real-World Multi-Modal Camera Tracking Benchmark

Sanghyun Woo, Kwanyong Park, Inkyu Shin, Myungchul Kim, In So Kweon

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted on CVPR 2024

RELI11D: A Comprehensive Multimodal Human Motion Dataset and Method

Ming Yan, Yan Zhang, Shuqiang Cai, Shuqi Fan, Xincheng Lin, Yudi Dai, Siqi Shen, Chenglu Wen, Lan Xu, Yuexin Ma, Cheng Wang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments CVPR2024, Project website: http://www.lidarhumanmotion.net/reli11d/

SpectralWaste Dataset: Multimodal Data for Waste Sorting Automation

Sara Casao, Fernando Peña, Alberto Sabater, Rosa Castillón, Darío Suárez, Eduardo Montijano, Ana C. Murillo

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Common Ground Tracking in Multimodal Dialogue

Ibrahim Khebour, Kenneth Lai, Mariah Bradford, Yifan Zhu, Richard Brutti, Christopher Tam, Jingxuan Tu, Benjamin Ibarra, Nathaniel Blanchard, Nikhil Krishnaswamy, James Pustejovsky

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

mForms : Multimodal Form-Filling with Question Answering

Larry Heck, Simon Heck, Anirudh Sundar

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Comments 5 pages, 6 figures, 4 tables

SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery

Xin Guo, Jiangwei Lao, Bo Dang, Yingying Zhang, Lei Yu, Lixiang Ru, Liheng Zhong, Ziyuan Huang, Kang Wu, Dingxiang Hu, Huimei He, Jian Wang, Jingdong Chen, Ming Yang, Yongjun Zhang, Yansheng Li

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR2024

GSVA: Generalized Segmentation via Multimodal Large Language Models

Zhuofan Xia, Dongchen Han, Yizeng Han, Xuran Pan, Shiji Song, Gao Huang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR2024 (19 pages, 9 figures, 11 tables)

PathMMU: A Massive Multimodal Expert-Level Benchmark for Understanding and Reasoning in Pathology

Yuxuan Sun, Hao Wu, Chenglu Zhu, Sunyi Zheng, Qizi Chen, Kai Zhang, Yunlong Zhang, Dan Wan, Xiaoxiao Lan, Mengyue Zheng, Jingxiong Li, Xinheng Lyu, Tao Lin, Lin Yang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments 27 pages, 12 figures

Federated Modality-specific Encoders and Multimodal Anchors for Personalized Brain Tumor Segmentation

Qian Dai, Dong Wei, Hong Liu, Jinghan Sun, Liansheng Wang, Yefeng Zheng

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by AAAI 2024

MASSTAR: A Multi-Modal and Large-Scale Scene Dataset with a Versatile Toolchain for Surface Prediction and Completion

Guiyong Zheng, Jinqi Jiang, Chen Feng, Shaojie Shen, Boyu Zhou

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Comments Submitted to IROS2024. Code: https://github.com/SYSU-STAR/MASSTAR. Project Page: https://github.com/SYSU-STAR/MASSTAR

Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Clouds

Tiago Cortinhal, Eren Erdal Aksoy

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV

Improving prediction of students' performance in intelligent tutoring systems using attribute selection and ensembles of different multimodal data sources

W. Chango, R. Cerezo, M. Sanchez-Santillan, R. Azevedo, C. Romero

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.AI

Journal ref Journal of Computing in Higher Education,2021, 33, 614-634

Exploiting Multimodal Synthetic Data for Egocentric Human-Object Interaction Detection in an Industrial Scenario

Rosario Leonardi, Francesco Ragusa, Antonino Furnari, Giovanni Maria Farinella

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Multimodal Transformer for Comics Text-Cloze

Emanuele Vivoli, Joan Lafuente Baeza, Ernest Valveny Llobet, Dimosthenis Karatzas

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV

Adding Multimodal Capabilities to a Text-only Translation Model

Vipin Vijayan, Braeden Bowen, Scott Grigsby, Timothy Anderson, Jeremy Gwinnup

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL

The Case for Evaluating Multimodal Translation Models on Text Datasets

Vipin Vijayan, Braeden Bowen, Scott Grigsby, Timothy Anderson, Jeremy Gwinnup

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL