arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.12980cs.LG
收藏

循环重构解纠缠

Disentanglement by Cyclic Reconstruction

IRT Saint-Exupéry(圣埃克苏佩里跨学科研究中心);ISAE-SUPAERO(法国高等航空航天学院);ANITI(图卢兹人工智能与自然智能研究所);Université de Toulouse(图卢兹大学)

David Bertoin, Emmanuel Rachelson

摘要:

深度神经网络已经证明了它们从数据中自动提取有意义特征的能力。然而,在监督学习中,特定于用于训练的数据集但与手头的任务无关的信息可以保持编码在提取的表示中。这些剩余的信息引入了特定领域的偏差,削弱了泛化性能。在这项工作中,我们建议分裂成一个任务相关的表示和它的互补上下文表示的信息。我们提出了一种新颖的方法,结合对抗性特征预测和循环重建,在单域监督的情况下解开这两种表示。然后,我们适应这种方法的无监督领域自适应问题,包括训练一个模型,能够执行的源和目标域。特别是,我们的方法促进了目标域的解纠缠,尽管没有训练标签。这使得特定于任务的信息能够从两个域中隔离出来,并投影到一个公共的表示中。特定于任务的表示允许将从源域获得的知识有效地转移到目标域。在单域的情况下,我们证明了我们的质量表示信息检索任务和泛化的好处引起的锐化特定任务的表示。然后,我们验证了几个经典的领域自适应基准所提出的方法,并说明了解纠缠的领域自适应的好处。

英文摘要:

Deep neural networks have demonstrated their ability to automatically extract meaningful features from data. However, in supervised learning, information specific to the dataset used for training, but irrelevant to the task at hand, may remain encoded in the extracted representations. This remaining information introduces a domain-specific bias, weakening the generalization performance. In this work, we propose splitting the information into a task-related representation and its complementary context representation. We propose an original method, combining adversarial feature predictors and cyclic reconstruction, to disentangle these two representations in the single-domain supervised case. We then adapt this method to the unsupervised domain adaptation problem, consisting of training a model capable of performing on both a source and a target domain. In particular, our method promotes disentanglement in the target domain, despite the absence of training labels. This enables the isolation of task-specific information from both domains and a projection into a common representation. The task-specific representation allows efficient transfer of knowledge acquired from the source domain to the target domain. In the single-domain case, we demonstrate the quality of our representations on information retrieval tasks and the generalization benefits induced by sharpened task-specific representations. We then validate the proposed method on several classical domain adaptation benchmarks and illustrate the benefits of disentanglement for domain adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑