arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.16554cs.CLcs.AI
收藏

PE:一种用于快速文本层级生成的 Poincaré 解释方法

PE: A Poincare Explanation Method for Fast Text Hierarchy Generation

School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);NPPA Key Laboratory of Publishing Integration Development, ECNUP(ECNUP NPPA出版融合发展重点实验室);Sichuan Caizi Software Information Network Co., Ltd(四川才子软件信息网络有限公司)

Qian Chen, Dongyang Li, Xiaofeng He, Hongzhao Li, Hongyu Yi

AI总结:

本文提出 PE 方法,将文本层级生成转化为双曲空间中的最小生成树构建,通过嵌入投影与 Shapley 分数实现高效特征交互建模。

摘要:

深度学习模型在 NLP 中的黑箱性质阻碍了其广泛应用。由于层次归因(Hierarchical Attribution,HA)能够建模特征交互,研究焦点已转向该方法。近期工作通过在欧几里得空间中进行耗时的贪婪搜索来建模非连续组合,却忽视了特征表示中潜在的语言信息。在本工作中,我们提出一种名为 Poincaré Explanation(PE)的新方法,以双曲空间高效地建模特征交互。具体而言,我们将构建文本层级视为在双曲空间中寻找生成树。首先,我们将嵌入投影到双曲空间,以引出固有的语义和句法层级结构。随后,我们提出一种简单而有效的策略来计算 Shapley 分数。最后,我们构建该层级,并证明投影空间中的构建过程可视为构建最小生成树,同时引入一种省时的构建算法。实验结果证明了本方法的有效性。

英文摘要:

The black-box nature of deep learning models in NLP hinders their widespread application. The research focus has shifted to Hierarchical Attribution (HA) for its ability to model feature interactions. Recent works model non-contiguous combinations with a time-costly greedy search in Eculidean spaces, neglecting underlying linguistic information in feature representations. In this work, we introduce a novel method, namely Poincare Explanation (PE), for modeling feature interactions with hyperbolic spaces in a time efficient manner. Specifically, we take building text hierarchies as finding spanning trees in hyperbolic spaces. First we project the embeddings into hyperbolic spaces to elicit inherit semantic and syntax hierarchical structures. Then we propose a simple yet effective strategy to calculate Shapley score. Finally we build the the hierarchy with proving the constructing process in the projected space could be viewed as building a minimum spanning tree and introduce a time efficient building algorithm. Experimental results demonstrate the effectiveness of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑