Cross-Modal Purification and Fusion for Small-Object RGB-D Transmission-Line Defect Detection
School of Mechatronics Engineering, Harbin Institute of Technology(机械电子工程学院,哈尔滨工业大学);School of Instrument Science and Engineering, Harbin Institute of Technology(仪器科学与工程学院,哈尔滨工业大学);Electric Power Research Institute, Yunnan Power Grid Co., Ltd.(电力研究院,云南电网公司)
Transmission line defect detection remains challenging for automated UAV inspection due to the dominance of small-scale defects, complex backgrounds, and illumination variations. Existing RGB-based detectors, despite recent progress, struggle to distinguish geometrically subtle defects from visually similar background structures under limited chromatic contrast. This paper proposes CMAFNet, a Cross-Modal Alignment and Fusion Network that integrates RGB appearance and depth geometry through a principled purify-then-fuse paradigm. CMAFNet consists of a Semantic Recomposition Module that performs dictionary-based feature purification via a learned codebook to suppress modality-specific noise while preserving defect-discriminative information, and a Contextual Semantic Integration Framework that captures global spatial dependencies using partial-channel attention to enhance structural semantic reasoning. Position-wise normalization within the purification stage enforces explicit reconstruction-driven cross-modal alignment, ensuring statistical compatibility between heterogeneous features prior to fusion. Extensive experiments on the TLRGBD benchmark, where 94.5% of instances are small objects, demonstrate that CMAFNet achieves 32.2% mAP@50 and 12.5% APs, outperforming the strongest baseline by 9.8 and 4.0 percentage points, respectively. A lightweight variant reaches 24.8% mAP50 at 228 FPS with only 4.9M parameters, surpassing all YOLO-based detectors while matching transformer-based methods at substantially lower computational cost.
TPCap: Unlocking Zero-Shot Image Captioning with Trigger-Augmented and Multi-Modal Purification Modules
Faculty of Information Engineering and Automation, Kunming University of Science and Technology(昆明理工大学信息工程与自动化学院);Yunnan Key Laboratory of Computer Technologies Application(云南省计算机技术应用重点实验室);Hongyun Honghe Group Honghe Cigarette Factory(红云红河集团红河卷烟厂)
Retrieving to Recover: Towards Incomplete Audio-Visual Question Answering via Semantic-consistent Purification
School of Computing and Information Technology, Great Bay University(大湾大学计算机与信息学院);Dongguan Key Laboratory for Intelligence and Information Technology(东莞智能与信息科技重点实验室);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);College of Computer Science, Nankai University(南开大学计算机学院);School of mathematics, Sun Yat-sen University(中山大学数学学院)
FusionRS: A Large-Scale RGB-Infrared-Style Remote Sensing Dataset for Cross-Modal Vision-Language Learning
China University of Petroleum-Beijing at Karamay(中国石油大学(北京)克拉玛依校区);University of Electronic Science and Technology of China(电子科技大学);Tianjin University(天津大学)
UASTrack: A Unified Adaptive Selection Framework with Modality-Customization in Single Object Tracking
School of Artificial Intelligence and Computer Science, Jiangnan University(江南大学人工智能与计算机科学学院);Centre for Vision, Speech and Signal Processing, University of Surrey(萨里大学视觉、语音和信号处理中心)
Cooperative Medianet Innovation Center(合作 medianet 创新中心);Shanghai Jiao Tong University(上海交通大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));CloudWalk Technology Co., Ltd(云从科技股份有限公司);Shanghai AI Laboratory(上海人工智能实验室)
Learning Progressive Adaptation for Multi-Modal Tracking
School of Artificial Intelligence and Computer Science, Jiangnan University(江南大学人工智能与计算机科学学院);Centre for Vision, Speech and Signal Processing, University of Surrey(Surrey大学视觉、语音和信号处理中心)
Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);Beijing Institute of Technology(北京理工大学);University of Chinese Academy of Sciences(中国科学院大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Peng Cheng Laboratory(鹏城实验室);School of Computer Science, Peking University(北京大学计算机学院);School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University(北京大学深圳研究生院电子与计算机工程学院)
CIR-Net: Cross-modality Interaction and Refinement for RGB-D Salient Object Detection
Beijing Jiaotong University(北京交通大学);Nanyang Technological University(南洋理工大学);Sun Yat-sen University(中山大学);University of Chinese Academy of Sciences(中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Peng Cheng Laboratory(鹏城实验室)
Great Bay University(大湾区大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);SailYond Technology(赛扬科技);University of Oulu(奥卢大学);Nanjing University of Information Science and Technology(南京信息工程大学)
Modality Dominance-Aware Optimization for Embodied RGB-Infrared Perception
College of Electronics and Information Engineering(电子信息工程学院);School of Computer Science and Technology(计算机科学与技术学院);Department of Electrical and Computer Engineering(电气与计算机工程系);School of Future Science and Engineering(未来科学与工程学院);Key Laboratory of General Artificial Intelligence and Large Models in Provincial Universities(省属大学通用人工智能与大模型重点实验室)
Dissecting RGB-D Learning for Improved Multi-modal Fusion
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及跨学科应用国家重点实验室(东南大学));Tsinghua University(清华大学);College of Computer Science, Beijing University of Technology(北京理工大学计算机学院)
Computer Vision Center, Universitat Autonoma de Barcelona(巴塞罗那自治大学计算机视觉中心);Computer Vision Laboratory, ETH Zürich(苏黎世联邦理工学院计算机视觉实验室);Inception Institute of Artificial Intelligence(inception人工智能研究院)
TUNI: Unifying Pre-training and Fine-tuning with Modality-Aware Mutual Learning and Rectification for RGB-T Semantic Segmentation
School of Automation, Beijing Institute of Technology(自动化学院,北京理工大学);School of Computer Science, Wuhan University(计算机学院,武汉大学);Great Wall Motor(长城汽车);School of Information and Electronic Engineering, Zhejiang University of Science and Technology(信息电子工程学院,浙江理工大学)
Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);University of Science and Technology of China(中国科学技术大学);Center for Excellence in Brain Science and Intelligence Technology(脑科学与智能技术卓越创新中心)
Decoupled Cross-Modal Alignment Network for Text-RGBT Person Retrieval and A High-Quality Benchmark
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);National Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology(光电信息采集与防护技术国家重点实验室);Anhui Provincial Key Laboratory of Multimodal Cognitive Computation(安徽省多模态认知计算重点实验室)
Robust RGB-T Tracking via Learnable Visual Fourier Prompt Fine-tuning and Modality Fusion Prompt Generation
Key Laboratory of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University(教育区块链与智能技术重点实验室,教育部,广西师范大学);Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University(多源信息挖掘与安全广西重点实验室,广西师范大学)
最近基于RGBD的显著性检测模型引起了研究的关注。深度线索,如边界线索,表面法线,形状属性等,有助于识别具有复杂场景的显著对象。然而,大多数RGBD网络需要输入端的多模态,并通过双流设计分别馈送它们,这不可避免地导致深度传感器和计算的额外成本。为了解决这些不便,本文提出了一种新的融合设计命名为模态引导子网络(MGSnet)。它具有以下优越的设计:1)我们的模型适用于RGB和RGBD数据,如果不可用,则动态估计深度。考虑到深度预测网络的内部工作原理,我们建议从RGB输入中估计伪几何图-基本上模仿多模态输入。2)我们的MGSnet for RGB SOD可实现实时推理,但与其他RGB模型相比,其性能达到了最先进的水平。3)MGS的灵活和轻量级设计有助于集成到RGBD双流模型中。引入的融合设计实现了跨模态交互,以实现进一步的进展,但成本最低。
RGB-D Salient Object Detection with Cross-Modality Modulation and Selection
Nanyang Technological University(南洋理工大学);Beijing Jiaotong University(北京交通大学);Dalian University of Technology(大连理工大学);Chinese Academy of Sciences(中国科学院)
RA-SOD: Reliability-Aware RGB-T Salient Object Detection under Modality Degradation
State Key Laboratory of Robotics and Systems, Harbin Institute of Technology(哈尔滨工业大学机器人技术与系统国家重点实验室);School of Computer Science, University of Sydney(悉尼大学计算机科学学院)