多模态大模型

PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection

Yidi Li, Jiahao Wen, Bin Ren, Wenhao Li, Zhenhuan Xu, Hao Guo, Hong Liu, Nicu Sebe

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments 3D Object Detection

Time Series Analysis for Education: Methods, Applications, and Future Directions

Shengzhong Mao, Chaoli Zhang, Yichi Song, Jindong Wang, Xiao-Jun Zeng, Zenglin Xu, Qingsong Wen

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Comments 24 pages, 3 figures, 6 tables, project page: see https://github.com/ai-for-edu/time-series-analysis-for-education

Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalities

Zheyu Zhang, Xinzhao Liu, Zheng Chen, Yueyi Zhang, Huanjing Yue, Yunwei Ou, Xiaoyan Sun

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted Paper to European Conference on Artificial Intelligence (ECAI 2024)

Hierarchical Attention and Parallel Filter Fusion Network for Multi-Source Data Classification

Han Luo, Feng Gao, Junyu Dong, Lin Qi

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by IEEE GRSL

Can Large Language Models Automatically Jailbreak GPT-4V?

Yuanwei Wu, Yue Huang, Yixin Liu, Xiang Li, Pan Zhou, Lichao Sun

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CL

Comments TrustNLP@NAACL2024 (Fourth Workshop on Trustworthy Natural Language Processing)

Molecular Graph Representation Learning Integrating Large Language Models with Domain-specific Small Models

Tianyu Zhang, Yuxiang Ren, Chengbin Hou, Hairong Lv, Xuegong Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI

Adaptive Learning of Consistency and Inconsistency Information for Fake News Detection

Aohan Li, Jiaxin Chen, Xin Liao, Dengyong Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

To Impute or Not: Recommendations for Multibiometric Fusion

Melissa R Dale, Elliot Singer, Bengt J. Borgström, Arun Ross

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments Proc. of IEEE International Workshop on Information Forensics and Security (WIFS), (Nuremberg, Germany), December 2023

Modality-Balanced Learning for Multimedia Recommendation

Jinghao Zhang, Guofan Liu, Qiang Liu, Shu Wu, Liang Wang

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments ACM Multimedia 2024 (Oral)

UNER: A Unified Prediction Head for Named Entity Recognition in Visually-rich Documents

Yi Tu, Chong Zhang, Ya Guo, Huan Chen, Jinyang Tang, Huijia Zhu, Qi Zhang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CL

Comments accepted by ACM Multimedia 2024

MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts

Xi Victoria Lin, Akshat Shrivastava, Liang Luo, Srinivasan Iyer, Mike Lewis, Gargi Ghosh, Luke Zettlemoyer, Armen Aghajanyan

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Comments v2 -> update related work section v3 -> fix spelling

Weakly Supervised LiDAR Semantic Segmentation via Scatter Image Annotation

Yilong Chen, Zongyi Xu, xiaoshui Huang, Ruicheng Zhang, Xinqi Jiang, Xinbo Gao

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations

Hongrong Cheng, Miao Zhang, Javen Qinfeng Shi

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments IEEE TPAMI major revision

LAC-Net: Linear-Fusion Attention-Guided Convolutional Network for Accurate Robotic Grasping Under the Occlusion

Jinyu Zhang, Yongchong Gu, Jianxiong Gao, Haitao Lin, Qiang Sun, Xinwei Sun, Xiangyang Xue, Yanwei Fu

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Comments accepted by IROS2024

Cross-modulated Attention Transformer for RGBT Tracking

Yun Xiao, Jiacong Zhao, Andong Lu, Chenglong Li, Yin Lin, Bing Yin, Cong Liu

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 5 figures

AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion

Mohamad Qadri, Kevin Zhang, Akshay Hinduja, Michael Kaess, Adithya Pediredla, Christopher A. Metzler

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Comments SIGGRAPH 2024 (conference track full paper). First two authors contributed equally. Paper website: https://aoneus.github.io/

MiPa: Mixed Patch Infrared-Visible Modality Agnostic Object Detection

Heitor R. Medeiros, David Latortue, Eric Granger, Marco Pedersoli

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment

Dongjae Shin, Hyeonseok Lim, Inho Won, Changsu Choi, Minjun Kim, Seungwoo Song, Hangyeol Yoo, Sangmin Kim, Kyungtae Lim

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CL

Journal ref https://aclanthology.org/2024.findings-naacl.158

BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining

Minjun Kim, Seungwoo Song, Youhan Lee, Haneol Jang, Kyungtae Lim

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CL

A Recent Survey of Heterogeneous Transfer Learning

Runxue Bao, Yiming Sun, Yuhe Gao, Jindong Wang, Qiang Yang, Zhi-Hong Mao, Ye Ye

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI

Comments 23 pages, 7 figures

SkipcrossNets: Adaptive Skip-cross Fusion for Road Detection

Yan Gong, Xinyu Zhang, Hao Liu, Xinmin Jiang, Zhiwei Li, Xin Gao, Lei Lin, Dafeng Jin, Jun Li, Huaping Liu

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

OmniSat: Self-Supervised Modality Fusion for Earth Observation

Guillaume Astruc, Nicolas Gonthier, Clement Mallet, Loic Landrieu

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV

Journal ref ECCV 2024

DVLO: Deep Visual-LiDAR Odometry with Local-to-Global Feature Fusion and Bi-Directional Structure Alignment

Jiuming Liu, Dong Zhuo, Zhiheng Feng, Siting Zhu, Chensheng Peng, Zhe Liu, Hesheng Wang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by ECCV 2024.Codes are released at https://github.com/IRMVLab/DVLO

MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillation

Xiaoshuai Hao, Ruikai Li, Hui Zhang, Dingzhe Li, Rong Yin, Sangil Jung, Seung-In Park, ByungIn Yoo, Haimei Zhao, Jing Zhang

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by ECCV2024

Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalities

Xu Zheng, Yuanhuiyi Lyu, Lin Wang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted to ECCV 2024

NODE-Adapter: Neural Ordinary Differential Equations for Better Vision-Language Reasoning

Yi Zhang, Chun-Wun Cheng, Ke Yu, Zhihai He, Carola-Bibiane Schönlieb, Angelica I. Aviles-Rivero

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sight

Hang Wu, Zhenghao Zhang, Siyuan Lin, Tong Qin, Jin Pan, Qiang Zhao, Chunjing Xu, Ming Yang

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Comments IEEE IV 2024

DMM: Disparity-guided Multispectral Mamba for Oriented Object Detection in Remote Sensing

Minghang Zhou, Tianyu Li, Chaofan Qiao, Dongyu Xie, Guoqing Wang, Ningjuan Ruan, Lin Mei, Yang Yang

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV

Comments 12 pages, 9 figures

OccFusion: Depth Estimation Free Multi-sensor Fusion for 3D Occupancy Prediction

Ji Zhang, Yiran Ding, Zixin Liu

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV

Fine-Grained Scene Image Classification with Modality-Agnostic Adapter

Yiqun Wang, Zhao Zhou, Xiangcheng Du, Xingjiao Wu, Yingbin Zheng, Cheng Jin

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV