arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.14276cs.CLcs.AIcs.LG
收藏

基于示例的 Hypernetwork 用于分布外泛化

Example-based Hypernetworks for Out-of-Distribution Generalization

Bar Ilan University(巴伊兰大学);Technion, IIT(以色列理工学院);NVIDIA Research(英伟达研究院)

Tomer Volk, Eyal Ben-David, Ohad Amosy, Gal Chechik, Roi Reichart

AI总结:

本文提出基于示例的 Hypernetwork 适应框架,利用多源领域标注数据在训练时泛化到未知目标领域,通过 T5 生成签名并由 Hypernetwork 生成分类器权重,在情感分类和自然语言推理任务中超越既有算法,首次将 Hypernetwork 应用于未知领域适应。

摘要:

随着自然语言处理(NLP)算法不断取得新的里程碑,分布外泛化仍然是一个重大挑战。本文针对不熟悉领域的多源适应问题:我们利用多个源领域的标注数据,在训练时泛化到未知目标领域。我们的创新框架采用基于示例的 Hypernetwork 适应:一个 T5 编码器-解码器首先从输入示例生成独特的签名,将其嵌入源领域的语义空间。随后,Hypernetwork 利用该签名生成任务分类器的权重。我们在两个任务——情感分类和自然语言推理——的 29 个适应场景中评估了该方法,其表现超越了既有算法。在高级版本中,签名还丰富了输入示例的表示。我们还将微调后的架构与少样本 GPT-3 进行比较,证明其在关键用例中的有效性。据我们所知,这标志着 Hypernetwork 首次应用于未知领域的适应。

英文摘要:

As Natural Language Processing (NLP) algorithms continually achieve new milestones, out-of-distribution generalization remains a significant challenge. This paper addresses the issue of multi-source adaptation for unfamiliar domains: We leverage labeled data from multiple source domains to generalize to unknown target domains at training. Our innovative framework employs example-based Hypernetwork adaptation: a T5 encoder-decoder initially generates a unique signature from an input example, embedding it within the source domains' semantic space. This signature is subsequently utilized by a Hypernetwork to generate the task classifier's weights. We evaluated our method across two tasks - sentiment classification and natural language inference - in 29 adaptation scenarios, where it outpaced established algorithms. In an advanced version, the signature also enriches the input example's representation. We also compare our finetuned architecture to few-shot GPT-3, demonstrating its effectiveness in essential use cases. To our knowledge, this marks the first application of Hypernetworks to the adaptation for unknown domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑