多模态大模型

Pan-Mamba: Effective pan-sharpening with State Space Model

Xuanhua He, Ke Cao, Keyu Yan, Rui Li, Chengjun Xie, Jie Zhang, Man Zhou

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Multi-View Causal Representation Learning with Partial Observability

Dingling Yao, Danru Xu, Sébastien Lachapelle, Sara Magliacane, Perouz Taslakian, Georg Martius, Julius von Kügelgen, Francesco Locatello

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Comments 28 pages, 10 figures, 11 tables

COMMIT: Certifying Robustness of Multi-Sensor Fusion Systems against Semantic Attacks

Zijian Huang, Wenda Chu, Linyi Li, Chejian Xu, Bo Li

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Unveiling Hidden Links Between Unseen Security Entities

Daniel Alfasi, Tal Shapira, Anat Bremler Barr

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine

Xiaosong Wang, Xiaofan Zhang, Guotai Wang, Junjun He, Zhongyu Li, Wentao Zhu, Yi Guo, Qi Dou, Xiaoxiao Li, Dequan Wang, Liang Hong, Qicheng Lao, Tong Ruan, Yukun Zhou, Yixue Li, Jie Zhao, Kang Li, Xin Sun, Lifeng Zhu, Shaoting Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Technical Report. Visit https://github.com/openmedlab for more details

Learning A Multi-Task Transformer Via Unified And Customized Instruction Tuning For Chest Radiograph Interpretation

Lijian Xu, Ziyu Ni, Xinglong Liu, Xiaosong Wang, Hongsheng Li, Shaoting Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Humanoid Locomotion as Next Token Prediction

Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasegaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, Jitendra Malik

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Explicit Interaction for Fusion-Based Place Recognition

Jingyi Xu, Junyi Ma, Qi Wu, Zijie Zhou, Yue Wang, Xieyuanli Chen, Ling Pei

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Parameter-efficient Prompt Learning for 3D Point Cloud Understanding

Hongyu Sun, Yongcai Wang, Wang Chen, Haoran Deng, Deying Li

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments 9 pages, 5 figures, 6 tables; accepted by ICRA 2024

The AffectToolbox: Affect Analysis for Everyone

Silvan Mertes, Dominik Schiller, Michael Dietz, Elisabeth André, Florian Lingenfelser

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

When Large Language Model Agents Meet 6G Networks: Perception, Grounding, and Alignment

Minrui Xu, Dusit Niyato, Jiawen Kang, Zehui Xiong, Shiwen Mao, Zhu Han, Dong In Kim, Khaled B. Letaief

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

VisIRNet: Deep Image Alignment for UAV-taken Visible and Infrared Image Pairs

Sedat Ozer, Alain P. Ndigande

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Neural Rendering based Urban Scene Reconstruction for Autonomous Driving

Shihao Shen, Louis Kerofsky, Varun Ravi Kumar, Senthil Yogamani

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted for publication in Electronic Imaging, Autonomous Vehicles and Machines 2024. Qualitative results are shared in https://youtu.be/EK47fYJiY3M

Efficient Multi-Resolution Fusion for Remote Sensing Data with Label Uncertainty

Hersh Vakharia, Xiaoxiao Du

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments 4 pages, 3 figures, 2 tables; Accepted to International Geoscience and Remote Sensing Symposium (IGARSS) 2023; Code available at https://github.com/hvak/MIMRF-BFM

CogVLM: Visual Expert for Pretrained Language Models

Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, Jie Tang

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Fisheye Camera and Ultrasonic Sensor Fusion For Near-Field Obstacle Perception in Bird's-Eye-View

Arindam Das, Sudarshan Paul, Niko Scholz, Akhilesh Kumar Malviya, Ganesh Sistu, Ujjwal Bhattacharya, Ciarán Eising

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments 16 pages, 12 Figures, 6 tables

Tensor-view Topological Graph Neural Network

Tao Wen, Elynn Chen, Yuzhou Chen

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Comments Accepted at AISTATS 2024

Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images

Hansa Srinivasan, Candice Schumann, Aradhana Sinha, David Madras, Gbolahan Oluwafemi Olanubi, Alex Beutel, Susanna Ricco, Jilin Chen

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CV

InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance

Pengyu Wang, Dong Zhang, Linyang Li, Chenkun Tan, Xinghao Wang, Ke Ren, Botian Jiang, Xipeng Qiu

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CL

Representation Learning for Wearable-Based Applications in the Case of Missing Data

Janosch Jungo, Yutong Xiang, Shkurta Gashi, Christian Holz

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Comments Paper accepted in Human-Centric Representation Learning workshop at AAAI 2024 (https://hcrl-workshop.github.io/2024/)

Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization

Jameel Hassan, Hanan Gani, Noor Hussein, Muhammad Uzair Khattak, Muzammal Naseer, Fahad Shahbaz Khan, Salman Khan

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2023

TrAISformer -- A Transformer Network with Sparse Augmented Data Representation and Cross Entropy Loss for AIS-based Vessel Trajectory Prediction

Duong Nguyen, Ronan Fablet

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

HawkRover: An Autonomous mmWave Vehicular Communication Testbed with Multi-sensor Fusion and Deep Learning

Ethan Zhu, Haijian Sun, Mingyue Ji

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments submitted to IEEE conferences for future publications

Modality Exchange Network for Retinogeniculate Visual Pathway Segmentation

Hua Han, Cheng Li, Lei Xie, Yuanjing Feng, Alou Diakite, Shanshan Wang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

MLIP: Medical Language-Image Pre-training with Masked Local Representation Learning

Jiarun Liu, Hong-Yu Zhou, Cheng Li, Weijian Huang, Hao Yang, Yong Liang, Shanshan Wang

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CV

Comments 5 pages, 3 figures

Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning

Kaibin Tian, Yanhua Cheng, Yi Liu, Xinglin Hou, Quan Chen, Han Li

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

LCPR: A Multi-Scale Attention-Based LiDAR-Camera Fusion Network for Place Recognition

Zijie Zhou, Jingyi Xu, Guangming Xiong, Junyi Ma

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by IEEE Robotics and Automation Letters (RAL) 2023

Joint Sensing and Task-Oriented Communications with Image and Wireless Data Modalities for Dynamic Spectrum Access

Yalin E. Sagduyu, Tugba Erpek, Aylin Yener, Sennur Ulukus

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Misalign, Contrast then Distill: Rethinking Misalignments in Language-Image Pretraining

Bumsoo Kim, Yeonsik Jo, Jinhyung Kim, Seung Hwan Kim

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CV

Comments ICCV 2023

Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders

Yaohua Zha, Huizhen Ji, Jinmin Li, Rongsheng Li, Tao Dai, Bin Chen, Zhi Wang, Shu-Tao Xia

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted to AAAI 2024