arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1702.08811stat.MLcs.LG
收藏

用于领域不变表示学习的中心矩离散算法(CMD)

Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning

Johannes Kepler University Linz(林茨约翰内斯·开普勒大学);Software Competence Center Hagenberg(哈根贝格软件能力中心)

Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, Susanne Saminger-Platz

摘要:

研究了在领域自适应的背景下,用神经网络学习领域不变表示的问题。我们提出了一种新的正则化方法,直接在隐藏的激活空间中最大限度地减少特定领域的潜在特征表示之间的差异。虽然存在一些标准的分布匹配方法,可以被解释为矩的加权和的匹配,例如最大均值离散(MMD),高阶矩的显式顺序匹配之前没有被考虑过。我们建议匹配高阶中心矩的概率分布的顺序明智的时刻差的手段。我们的模型不需要计算昂贵的距离和核矩阵计算。利用矩序列对概率分布的等价表示,定义了一种新的距离函数,称为中心矩离散(CMD)。证明了CMD是紧区间上概率分布集合上的度量。进一步证明了紧区间上的概率分布的收敛性。新的度量意味着各个随机变量的分布依分布收敛。我们在两个不同的基准数据集上测试我们的方法,用于对象识别(Office)和产品评论的情感分析(Amazon评论)。CMD在Office的大多数域自适应任务上实现了最先进的性能,并且在Amazon评论上优于使用MMD、变分公平自动编码器和域对抗神经网络训练的网络。此外,事后参数敏感性分析表明,新的方法是稳定的w.r.t.参数在一定的时间间隔内变化。实验的源代码是公开的。

英文摘要:

The learning of domain-invariant representations in the context of domain adaptation with neural networks is considered. We propose a new regularization method that minimizes the discrepancy between domain-specific latent feature representations directly in the hidden activation space. Although some standard distribution matching approaches exist that can be interpreted as the matching of weighted sums of moments, e.g. Maximum Mean Discrepancy (MMD), an explicit order-wise matching of higher order moments has not been considered before. We propose to match the higher order central moments of probability distributions by means of order-wise moment differences. Our model does not require computationally expensive distance and kernel matrix computations. We utilize the equivalent representation of probability distributions by moment sequences to define a new distance function, called Central Moment Discrepancy (CMD). We prove that CMD is a metric on the set of probability distributions on a compact interval. We further prove that convergence of probability distributions on compact intervals w.r.t. the new metric implies convergence in distribution of the respective random variables. We test our approach on two different benchmark data sets for object recognition (Office) and sentiment analysis of product reviews (Amazon reviews). CMD achieves a new state-of-the-art performance on most domain adaptation tasks of Office and outperforms networks trained with MMD, Variational Fair Autoencoders and Domain Adversarial Neural Networks on Amazon reviews. In addition, a post-hoc parameter sensitivity analysis shows that the new approach is stable w.r.t. parameter changes in a certain interval. The source code of the experiments is publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑