多模态大模型

SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation

Wenchao Sun, Xuewu Lin, Yining Shi, Chuang Zhang, Haoran Wu, Sifa Zheng

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

WTTFNet: A Weather-Time-Trajectory Fusion Network for Pedestrian Trajectory Prediction in Urban Complex

Ho Chun Wu, Esther Hoi Shan Lau, Paul Yuen, Kevin Hung, John Kwok Tai Chui, Andrew Kwok Fai Lui

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments 12 pages, 7 figures

Evaluating Zero-Shot GPT-4V Performance on 3D Visual Question Answering Benchmarks

Simranjit Singh, Georgios Pavlakos, Dimitrios Stamoulis

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted at 1st Workshop on Multimodalities for 3D Scenes CVPR 2024

Feasibility and benefits of joint learning from MRI databases with different brain diseases and modalities for segmentation

Wentian Xu, Matthew Moffat, Thalia Seale, Ziyun Liang, Felix Wagner, Daniel Whitehouse, David Menon, Virginia Newcombe, Natalie Voets, Abhirup Banerjee, Konstantinos Kamnitsas

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted to MIDL 2024

Journal ref Proceedings of Machine Learning Research, MIDL 2024

Efficient Remote Sensing with Harmonized Transfer Learning and Modality Alignment

Tengjun Huang

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by the Twelfth International Conference on Learning Representations (ICLR) Workshop

Adapting LLaMA Decoder to Vision Transformer

Jiahao Wang, Wenqi Shao, Mengzhao Chen, Chengyue Wu, Yong Liu, Taiqiang Wu, Kaipeng Zhang, Songyang Zhang, Kai Chen, Ping Luo

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments 23 pages, 11 figures

Multimodality Invariant Learning for Multimedia-Based New Item Recommendation

Haoyue Bai, Le Wu, Min Hou, Miaomiao Cai, Zhuangzhuang He, Yuyang Zhou, Richang Hong, Meng Wang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Towards Precision Healthcare: Robust Fusion of Time Series and Image Data

Ali Rasekh, Reza Heidari, Amir Hosein Haji Mohammad Rezaie, Parsa Sharifi Sedeh, Zahra Ahmadi, Prasenjit Mitra, Wolfgang Nejdl

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

ContextualFusion: Context-Based Multi-Sensor Fusion for 3D Object Detection in Adverse Operating Conditions

Shounak Sural, Nishad Sahu, Ragunathan Rajkumar

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments 8 pages, 8 figures

Masked Image Modelling for retinal OCT understanding

Theodoros Pissas, Pablo Márquez-Neila, Sebastian Wolf, Martin Zinkernagel, Raphael Sznitman

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

GLaD: Synergizing Molecular Graphs and Language Descriptors for Enhanced Power Conversion Efficiency Prediction in Organic Photovoltaic Devices

Thao Nguyen, Tiara Torres-Flores, Changhyun Hwang, Carl Edwards, Ying Diao, Heng Ji

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Comments In progress

Weakly supervised alignment and registration of MR-CT for cervical cancer radiotherapy

Jjahao Zhang, Yin Gu, Deyu Sun, Yuhua Gao, Ming Gao, Ming Cui, Teng Zhang, He Ma

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Self-Supervised Modality-Agnostic Pre-Training of Swin Transformers

Abhiroop Talasila, Maitreya Maity, U. Deva Priyakumar

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Hierarchical SegNet with Channel and Context Attention for Accurate Lung Segmentation in Chest X-ray Images

Mohammad Ali Labbaf Khaniki, Nazanin Mahjourian, Mohammad Manthouri

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

GestFormer: Multiscale Wavelet Pooling Transformer Network for Dynamic Hand Gesture Recognition

Mallika Garg, Debashis Ghosh, Pyari Mohan Pradhan

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Protein Representation Learning by Capturing Protein Sequence-Structure-Function Relationship

Eunji Ko, Seul Lee, Minseon Kim, Dongki Kim

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Comments ICLR 2024 MLGenX Workshop (Spotlight)

Multi-Object Tracking with Camera-LiDAR Fusion for Autonomous Driving

Riccardo Pieroni, Simone Specchia, Matteo Corno, Sergio Matteo Savaresi

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Published at IEEE European Control Conference 2024

Mesh Denoising Transformer

Wenbo Zhao, Xianming Liu, Deming Zhai, Junjun Jiang, Xiangyang Ji

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Middle Fusion and Multi-Stage, Multi-Form Prompts for Robust RGB-T Tracking

Qiming Wang, Yongqiang Bai, Hongxing Song

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images

Junwen Huang, Hao Yu, Kuan-Ting Yu, Nassir Navab, Slobodan Ilic, Benjamin Busam

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

ELiTe: Efficient Image-to-LiDAR Knowledge Transfer for Semantic Segmentation

Zhibo Zhang, Ximing Yang, Weizhong Zhang, Cheng Jin

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Comments 9 pages, 6 figures, ICME 2024 oral

AFter: Attention-based Fusion Router for RGBT Tracking

Andong Lu, Wanyu Wang, Chenglong Li, Jin Tang, Bin Luo

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Peer review

Fully Sparse Fusion for 3D Object Detection

Yingyan Li, Lue Fan, Yang Liu, Zehao Huang, Yuntao Chen, Naiyan Wang, Zhaoxiang Zhang

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments TPMAI 2024

Aligning Knowledge Graph with Visual Perception for Object-goal Navigation

Nuo Xu, Wen Wang, Rong Yang, Mengjie Qin, Zheyuan Lin, Wei Song, Chunlong Zhang, Jason Gu, Chao Li

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted to ICRA 2024

Machine Learning Techniques for MRI Data Processing at Expanding Scale

Taro Langner

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Book chapter pre-print

General Item Representation Learning for Cold-start Content Recommendations

Jooeun Kim, Jinri Kim, Kwangeun Yeo, Eungi Kim, Kyoung-Woon On, Jonghwan Mun, Joonseok Lee

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.MM

Comments 14 pages

ProTA: Probabilistic Token Aggregation for Text-Video Retrieval

Han Fang, Xianghao Zang, Chao Ban, Zerun Feng, Lanxiang Zhou, Zhongjiang He, Yongxiang Li, Hao Sun

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

UniM-OV3D: Uni-Modality Open-Vocabulary 3D Scene Understanding with Fine-Grained Feature Representation

Qingdong He, Jinlong Peng, Zhengkai Jiang, Kai Wu, Xiaozhong Ji, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Mingang Chen, Yunsheng Wu

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by IJCAI 2024

Pretraining Billion-scale Geospatial Foundational Models on Frontier

Aristeidis Tsaris, Philipe Ambrozio Dias, Abhishek Potnis, Junqi Yin, Feiyi Wang, Dalton Lunga

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Equivariant Multi-Modality Image Fusion

Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Kai Zhang, Shuang Xu, Dongdong Chen, Radu Timofte, Luc Van Gool

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2024