arXiv 2603.174742026-03-18cs.CVcs.AI

重新审视交叉注意力机制：利用有益噪声进行领域自适应学习

Revisiting Cross-Attention Mechanisms: Leveraging Beneficial Noise for Domain-Adaptive Learning

School of Engineering, Westlake University（西lake大学工程学院）；Centre for Artificial Intelligence and Robotics (CAIR), Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences (HKISI-CAS)（香港科学院人工智能与机器人中心）；State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Alibaba Group（阿里巴巴集团）

Zelin Zang, Yehui Yang, Fei Wang, Liangyu Li, Baigui Sun

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出DACSM框架，通过引入有益噪声增强交叉注意力，结合领域翻译和尺度感知对齐，提升跨领域表示学习的鲁棒性，实验显示在VisDA-2017等数据集上取得最佳性能。

摘要：

无监督领域自适应（UDA）旨在将知识从标记源领域转移到未标记目标领域，但常面临严重的领域和尺度差距，影响性能。现有基于交叉注意力的Transformer可以对齐领域特征，但难以在大尺度和外观变化下保持内容语义。为此，我们引入有益噪声的概念，通过注入受控扰动正则化交叉注意力，促使模型忽略风格干扰，专注于内容。我们提出领域自适应交叉尺度匹配（DACSM）框架，包括用于分离领域共享内容与领域特定风格的领域自适应Transformer（DAT），以及确保多分辨率下语义一致性的交叉尺度匹配（CSM）模块。DAT通过有益噪声增强交叉注意力，实现逐步领域翻译，获得内容一致且风格不变的表示。CSM确保尺度变化下的语义一致性。在VisDA-2017、Office-Home和DomainNet上的广泛实验表明，DACSM在VisDA-2017上比CDTrans提升高达+2.3%，并在挑战性“卡车”类别上提升+5.9%，证明有益噪声在处理尺度差异方面的有效性。这些结果突显了结合领域翻译、有益噪声增强的注意力和尺度感知对齐在稳健跨领域表示学习中的有效性。

英文摘要：

Unsupervised Domain Adaptation (UDA) seeks to transfer knowledge from a labeled source domain to an unlabeled target domain but often suffers from severe domain and scale gaps that degrade performance. Existing cross-attention-based transformers can align features across domains, yet they struggle to preserve content semantics under large appearance and scale variations. To explicitly address these challenges, we introduce the concept of beneficial noise, which regularizes cross-attention by injecting controlled perturbations, encouraging the model to ignore style distractions and focus on content. We propose the Domain-Adaptive Cross-Scale Matching (DACSM) framework, which consists of a Domain-Adaptive Transformer (DAT) for disentangling domain-shared content from domain-specific style, and a Cross-Scale Matching (CSM) module that adaptively aligns features across multiple resolutions. DAT incorporates beneficial noise into cross-attention, enabling progressive domain translation with enhanced robustness, yielding content-consistent and style-invariant representations. Meanwhile, CSM ensures semantic consistency under scale changes. Extensive experiments on VisDA-2017, Office-Home, and DomainNet demonstrate that DACSM achieves state-of-the-art performance, with up to +2.3% improvement over CDTrans on VisDA-2017. Notably, DACSM achieves a +5.9% gain on the challenging "truck" class of VisDA, evidencing the strength of beneficial noise in handling scale discrepancies. These results highlight the effectiveness of combining domain translation, beneficial-noise-enhanced attention, and scale-aware alignment for robust cross-domain representation learning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。