arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.01539cs.CVcs.AIcs.LG
收藏

DINE:基于单个和多个黑箱预测器的领域自适应

DINE: Domain Adaptation from Single and Multiple Black-box Predictors

CASIA(中国科学院自动化研究所);NUS(新加坡国立大学);ByteDance(字节跳动);UCAS(中国科学院大学);CEBSIT, CAS(中国科学院脑科学与智能技术卓越创新中心)

Jian Liang, Dapeng Hu, Jiashi Feng, Ran He

摘要:

为了减轻标记的负担,无监督领域自适应(UDA)旨在将先前和相关标记数据集(源)中的知识转移到新的未标记数据集(目标)。尽管取得了令人印象深刻的进展,但现有方法总是需要访问原始源数据并开发数据依赖的比对方法以以转导学习方式识别目标样本,这可能会引起源个体的隐私问题。最近的几项研究通过利用来自源域的训练有素的白盒模型来寻求替代解决方案,但它仍然可能通过生成对抗学习泄漏原始数据。本文研究了UDA的一个实用而有趣的设置,其中只有黑盒源模型(即,仅网络预测可用)在目标域中的自适应期间被提供。为了解决这个问题,我们提出了一个新的两步知识适应框架称为DIstill和fine-tuNE(DINE)。考虑到目标数据结构,DINE首先从源预测器中提取知识到定制的目标模型,然后微调提取的模型以进一步适应目标域。此外,神经网络不需要在DINE中跨域相同,甚至允许在低资源设备上进行有效适配。三种UDA情景的经验结果(即,单源、多源和部分集)证实,与最先进的数据依赖方法相比,DINE实现了极具竞争力的性能。代码可在\url{https://github.com/tim-learn/DINE/}上获得。

英文摘要:

To ease the burden of labeling, unsupervised domain adaptation (UDA) aims to transfer knowledge in previous and related labeled datasets (sources) to a new unlabeled dataset (target). Despite impressive progress, prior methods always need to access the raw source data and develop data-dependent alignment approaches to recognize the target samples in a transductive learning manner, which may raise privacy concerns from source individuals. Several recent studies resort to an alternative solution by exploiting the well-trained white-box model from the source domain, yet, it may still leak the raw data through generative adversarial learning. This paper studies a practical and interesting setting for UDA, where only black-box source models (i.e., only network predictions are available) are provided during adaptation in the target domain. To solve this problem, we propose a new two-step knowledge adaptation framework called DIstill and fine-tuNE (DINE). Taking into consideration the target data structure, DINE first distills the knowledge from the source predictor to a customized target model, then fine-tunes the distilled model to further fit the target domain. Besides, neural networks are not required to be identical across domains in DINE, even allowing effective adaptation on a low-resource device. Empirical results on three UDA scenarios (i.e., single-source, multi-source, and partial-set) confirm that DINE achieves highly competitive performance compared to state-of-the-art data-dependent approaches. Code is available at \url{https://github.com/tim-learn/DINE/}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑