arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1910.05862cs.LGstat.ML
收藏

嵌入的约束非仿射对齐

Constrained Non-Affine Alignment of Embeddings

University of Utah(犹他大学);Visa Research(Visa研究院)

Yuwei Wang, Yan Zheng, Yanqing Peng, Chin-Chia Michael Yeh, Zhongfang Zhuang, Das Mahashweta, Bendre Mangesh, Feifei Li, Wei Zhang, Jeff M. Phillips

摘要:

嵌入是数据分析任务的基本构建块之一。嵌入已经是大语言模型和图像分析的基本工具,并且它们的使用正在扩展到许多其他研究领域。这些分布式表示的生成通常是一个数据和计算昂贵的过程;然而,在它们被创建之后对其进行整体分析和调整仍然是一个发展中的领域。在本文中,我们首先提出了一个非常通用的定量测量嵌入数据中的特征的存在的基础上,如果它可以学习。然后,我们设计了一种方法来删除或减轻嵌入中不需要的功能,同时保留数据的基本结构。我们使用域对抗网络(DAN)来生成非仿射变换,但我们添加了约束以确保嵌入的基本结构得到保留。我们的实证结果表明,该算法显着优于国家的最先进的无监督算法在几个数据集,包括新的应用程序从行业。

英文摘要:

Embeddings are one of the fundamental building blocks for data analysis tasks. Embeddings are already essential tools for large language models and image analysis, and their use is being extended to many other research domains. The generation of these distributed representations is often a data- and computation-expensive process; yet the holistic analysis and adjustment of them after they have been created is still a developing area. In this paper, we first propose a very general quantitatively measure for the presence of features in the embedding data based on if it can be learned. We then devise a method to remove or alleviate undesired features in the embedding while retaining the essential structure of the data. We use a Domain Adversarial Network (DAN) to generate a non-affine transformation, but we add constraints to ensure the essential structure of the embedding is preserved. Our empirical results demonstrate that the proposed algorithm significantly outperforms the state-of-art unsupervised algorithm on several data sets, including novel applications from the industry.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑