arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.15912cs.CL
收藏

以完美保真度解释词嵌入:研究影响力预测案例研究

Explaining word embeddings with perfect fidelity: Case study in research impact prediction

Department of Econometrics, Prague University of Economics and Business(经济计量学系,布拉格经济大学);Environmental Genomics and Systems Biology Division, Lawrence Berkeley National Laboratory(环境基因组与系统生物学部,劳伦斯伯克利国家实验室);Department of System Analysis, Prague University of Economics and Business(系统分析系,布拉格经济大学);Department of Information and Knowledge Engineering, Prague University of Economics and Business(信息与知识工程系,布拉格经济大学)

Lucie Dvorackova, Marcin P. Joachimiak, Michal Cerny, Adriana Kubecova, Vilem Sklenak, Tomas Kliegr

AI总结:

针对词嵌入逻辑回归分类器解释保真度不足的问题,本文提出 SMER 特征重要性方法,并在 CORD-19 的 5 万篇论文上证明其具有完美理论保真度且优于 LIME、SHAP 等方法。

摘要:

用于学术文献质量预测的最佳性能方法均基于嵌入模型。除了在分类器中使用时的性能外,嵌入模型甚至还能对分类模型标注训练数据中未包含的词语给出预测,这在研究术语不断演变的背景下非常重要。尽管诸如 Local interpretable model-agnostic explanations(LIME,局部可解释模型无关解释)等模型无关解释方法可用于解释在嵌入模型上训练的机器学习分类器,但它们产生的结果与模型本身的对应关系令人怀疑。我们提出了一种新的特征重要性方法 Self-model Rated Entities(SMER,自模型评级实体),用于在词嵌入上训练的、基于逻辑回归的分类模型。我们证明 SMER 与被解释模型具有理论上完美的保真度,因为单个词语的 SMER 分数 logit 的平均值(即 SMER 解释)恰好对应于被解释模型预测的 logit。我们通过在来自 CORD-19 语料库的 50,000 篇研究文章(论文)上开展五个多样化实验,进行了定量和定性评估。通过 AOPC 曲线分析,我们通过实验证明 SMER 产生的解释优于 LIME、SHAP 和全局树代理模型。

英文摘要:

The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-model Rated Entities (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. Through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑