多模态大模型

MolSpectra: Pre-training 3D Molecular Representation with Multi-modal Energy Spectra

Liang Wang, Shaozhen Liu, Yu Rong, Deli Zhao, Qiang Liu, Shu Wu, Liang Wang

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.AI

Comments Accepted by ICLR 2025

PedDet: Adaptive Spectral Optimization for Multimodal Pedestrian Detection

Rui Zhao, Zeyu Zhang, Yi Xu, Yi Yao, Yan Huang, Wenxin Zhang, Zirui Song, Xiuying Chen, Yang Zhao

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CV

ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentation

Yuheng Xue, Nenglun Chen, Jun Liu, Wenyun Sun

专题命中 多模态训练与对齐 ：cross-modal(title)；分类 cs.CV

Comments 2025 International Conference on 3D Vision (3DV)

Cross-Modal Synergies: Unveiling the Potential of Motion-Aware Fusion Networks in Handling Dynamic and Static ReID Scenarios

Fuxi Ling, Hongye Liu, Guoqiang Huang, Jing Li, Hong Wu, Zhihao Tang

专题命中 多模态训练与对齐 ：cross-modal(title)；分类 cs.CV

MetaMorph: Multimodal Understanding and Generation via Instruction Tuning

Shengbang Tong, David Fan, Jiachen Zhu, Yunyang Xiong, Xinlei Chen, Koustuv Sinha, Michael Rabbat, Yann LeCun, Saining Xie, Zhuang Liu

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CV

Comments Project page at tsb0601.github.io/metamorph

MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity

Yangzhou Liu, Yue Cao, Zhangwei Gao, Weiyun Wang, Zhe Chen, Wenhai Wang, Hao Tian, Lewei Lu, Xizhou Zhu, Tong Lu, Yu Qiao, Jifeng Dai

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Comments 18 pages, 8 figures, technical report

Journal ref Sci China Inf Sci, 2024

Human-Robot Dialogue Annotation for Multi-Modal Common Ground

Claire Bonial, Stephanie M. Lukin, Mitchell Abrams, Anthony Baker, Lucia Donatelli, Ashley Foots, Cory J. Hayes, Cassidy Henry, Taylor Hudson, Matthew Marge, Kimberly A. Pollard, Ron Artstein, David Traum, Clare R. Voss

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CL

Comments 52 pages, 14 figures

Journal ref Language Resources and Evaluation 2024

VAIR: Visuo-Acoustic Implicit Representations for Low-Cost, Multi-Modal Transparent Surface Reconstruction in Indoor Scenes

Advaith V. Sethuraman, Onur Bagoren, Harikrishnan Seetharaman, Dalton Richardson, Joseph Taylor, Katherine A. Skinner

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Comments https://umfieldrobotics.github.io/VAIR_site/

Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusion

Minkyoung Cho, Yulong Cao, Jiachen Sun, Qingzhao Zhang, Marco Pavone, Jeong Joon Park, Heng Yang, Z. Morley Mao

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Comments 23 pages

Multimodal Clickbait Detection by De-confounding Biases Using Causal Representation Inference

Jianxing Yu, Shiqi Wang, Han Yin, Zhenlong Sun, Ruobing Xie, Bo Zhang, Yanghui Rao

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.AI

Deep Multimodal Fusion for Semantic Segmentation of Remote Sensing Earth Observation Data

Ivica Dimitrovski, Vlatko Spasev, Ivan Kitanovski

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CV

EEPNet: Efficient Edge Pixel-based Matching Network for Cross-Modal Dynamic Registration between LiDAR and Camera

Yuanchao Yue, Hui Yuan, Suai Li, Qi Jiang

专题命中 多模态训练与对齐 ：cross-modal(title)；分类 cs.CV

DIAL: Dense Image-text ALignment for Weakly Supervised Semantic Segmentation

Soojin Jang, Jungmin Yun, Junehyoung Kwon, Eunju Lee, Youngbin Kim

专题命中 多模态训练与对齐 ：image-text(title)；分类 cs.CV

Comments accepted by the European Conference on Computer Vision (ECCV), 2024

ProgramAlly: Creating Custom Visual Access Programs via Multi-Modal End-User Programming

Jaylin Herskovitz, Andi Xu, Rahaf Alharbi, Anhong Guo

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.AI

Comments UIST 2024

Multi-Modal Dialogue State Tracking for Playing GuessWhich Game

Wei Pang, Ruixue Duan, Jinfu Yang, Ning Li

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.AI

Comments Published at CICAI 2023 (CAAI-A), codes at https://github.com/xubuvd/GuessWhich

Retinal IPA: Iterative KeyPoints Alignment for Multimodal Retinal Imaging

Jiacheng Wang, Hao Li, Dewei Hu, Rui Xu, Xing Yao, Yuankai K. Tao, Ipek Oguz

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CV

Multi-modal Data Binding for Survival Analysis Modeling with Incomplete Data and Annotations

Linhao Qu, Dan Huang, Shaoting Zhang, Xiaosong Wang

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Comments Accepted by MICCAI 2024

GaitMA: Pose-guided Multi-modal Feature Fusion for Gait Recognition

Fanxu Min, Shaoxiang Guo, Fan Hao, Junyu Dong

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Comments Accepted to ICME 2024

Cross-Modal Spherical Aggregation for Weakly Supervised Remote Sensing Shadow Removal

Kaichen Chi, Wei Jing, Junjie Li, Qiang Li, Qi Wang

专题命中 多模态训练与对齐 ：cross-modal(title)；分类 cs.CV

Comments 9pages, 11 figures

Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals

Zhicheng Cai

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CV

PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology

George Shaikovski, Adam Casson, Kristen Severson, Eric Zimmermann, Yi Kan Wang, Jeremy D. Kunz, Juan A. Retamero, Gerard Oakley, David Klimstra, Christopher Kanan, Matthew Hanna, Michal Zelechowski, Julian Viret, Neil Tenenholtz, James Hall, Nicolo Fusi, Razik Yousfi, Peter Hamilton, William A. Moye, Eugene Vorontsov, Siqi Liu, Thomas J. Fuchs

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

MaskFuser: Masked Fusion of Joint Multi-Modal Tokenization for End-to-End Autonomous Driving

Yiqun Duan, Xianda Guo, Zheng Zhu, Zhen Wang, Yu-Kai Wang, Chin-Teng Lin

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizer

Zhen Zhao, Jingqun Tang, Chunhui Lin, Binghong Wu, Can Huang, Hao Liu, Xin Tan, Zhizhong Zhang, Yuan Xie

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Comments Accepted to CVPR2024

FUELVISION: A Multimodal Data Fusion and Multimodel Ensemble Algorithm for Wildfire Fuels Mapping

Riyaaz Uddien Shaik, Mohamad Alipour, Eric Rowell, Bharathan Balaji, Adam Watts, Ertugrul Taciroglu

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CV

Comments 40 pages

Test-time Distribution Learning Adapter for Cross-modal Visual Reasoning

Yi Zhang, Ce Zhang

专题命中 多模态训练与对齐 ：cross-modal(title)；分类 cs.CV

Comments Accepted by ICASSP 2024

Causality-based Cross-Modal Representation Learning for Vision-and-Language Navigation

Liuyi Wang, Zongtao He, Ronghao Dang, Huiyi Chen, Chengju Liu, Qijun Chen

专题命中 多模态训练与对齐 ：cross-modal(title)；分类 cs.CV

Comments 16 pages

Serial fusion of multi-modal biometric systems

Gian Luca Marcialis, Paolo Mastinu, Fabio Roli

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Journal ref IEEE International Workshop on Biometric Measurements and Systems for Security and Medical Applications (BioMS2010), September, 9, 2010, Taranto (Italy), ISBN: 978-1-4244-6302-2

GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection

Ziying Song, Haiyue Wei, Lin Bai, Lei Yang, Caiyan Jia

专题命中 多模态训练与对齐 ：multi-modal(title)；分类 cs.CV

Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignment

Cong-Duy Nguyen, The-Anh Vu-Le, Thong Nguyen, Tho Quan, Luu Anh Tuan

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CL

Multimodal Pretraining of Medical Time Series and Notes

Ryan King, Tianbao Yang, Bobak Mortazavi

专题命中 多模态训练与对齐 ：multimodal(title)；分类 cs.CL