Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Xiaomi Inc.(小米公司)
Xin Cong, Bowen Yu, Tingwen Liu, Shiyao Cui, Hengzhu Tang, Bin Wang
Few-shot classification tends to struggle when it needs to adapt to diverse domains. Due to the non-overlapping label space between domains, the performance of conventional domain adaptation is limited. Previous work tackles the problem in a transductive manner, by assuming access to the full set of test data, which is too restrictive for many real-world applications. In this paper, we set out to tackle this issue by introducing a inductive framework, DaFeC, to improve Domain adaptation performance for Few-shot classification via Clustering. We first build a representation extractor to derive features for unlabeled data from the target domain (no test data is necessary) and then group them with a cluster miner. The generated pseudo-labeled data and the labeled source-domain data are used as supervision to update the parameters of the few-shot classifier. In order to derive high-quality pseudo labels, we propose a Clustering Promotion Mechanism, to learn better features for the target domain via Similarity Entropy Minimization and Adversarial Distribution Alignment, which are combined with a Cosine Annealing Strategy. Experiments are performed on the FewRel 2.0 dataset. Our approach outperforms previous work with absolute gains (in classification accuracy) of 4.95%, 9.55%, 3.99% and 11.62%, respectively, under four few-shot settings.
Unsupervised Adversarial Domain Adaptation for Implicit Discourse Relation Classification
The University of Texas at Austin(德克萨斯大学奥斯汀分校)
隐性话语关系比显性话语关系更难分类,也更难诠释。我们解决了缺乏隐式关系训练数据的情况,并从显式关系中利用领域自适应(Ji et al.,2015年)。我们提出了一个无监督的对抗域自适应网络,配备了重建组件。我们的系统优于以前的作品和其他对抗基准的无监督领域自适应。此外,我们扩展我们的系统,以利用标记的数据,如果有的话。
On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies
Stanford University(斯坦福大学)
我们研究了如何以无监督的方式掩蔽和预测令牌可以产生语言结构和下游性能增益。最近的理论表明,预训练的语言模型通过隐含地充当下游任务的完形填空缩减的掩码来获得有用的归纳偏差。虽然有吸引力,我们表明,在实践中使用的随机掩蔽策略的成功,不能解释这样的完形填空式的面具。我们使用三个不同分类数据集的特定任务词典构建了类似完形填空的掩码,并表明大多数预训练的性能增益来自与词典无关的通用掩码。为了解释这些通用面具的经验成功,我们证明了掩蔽语言模型(MLM)的目标和现有的方法之间的对应关系,学习统计依赖的图形模型。使用此,我们推导出一种方法,用于提取这些学习的统计依赖MLMs,并表明这些依赖编码有用的归纳偏见的形式的句法结构。在无监督解析评估中,简单地在隐含的统计依赖结构上形成最小生成树优于无监督解析的经典方法(58.74 vs. 55.91 UUAS)。
Unsupervised neural adaptation model based on optimal transport for spoken language identification
National Institute of Information and Communications Technology(日本国立信息与通信技术研究所);Research Center for Information Technology Innovation, Academic Sinica(中央研究院资讯科技创新研究中心)