arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.07132cs.LGcs.CL
收藏

用于可解释机器学习的基于大语言模型的文本特征生成

LLM-based feature generation from text for interpretable machine learning

Vojtěch Balek, Lukáš Sýkora, Vilém Sklenák, Tomáš Kliegr

AI总结:

本研究提出利用大语言模型从文本中提取少量可解释特征用于机器学习,在两个科研数据集上实现了与前沿嵌入模型相当的预测性能,且特征可解释、方法可跨领域泛化。

摘要:

现有文本表示(如嵌入和词袋模型)因维度高、特征层面可解释性缺失或存疑,并不适用于规则学习。本文探究大语言模型(LLM)能否通过从文本中提取少量可解释特征来解决这一问题。我们在两个包含数千篇多学科科研文章、目标变量为科研影响力代理指标的数据集(CORD-19和M17+)上验证了这一过程。基于与科研影响力统计显著相关性的测试评估表明,LLama 2生成的特征具有语义意义。我们随后将这些生成的特征用于文本分类,预测CORD-19数据集上代表引用率的二元目标变量,以及M17+数据集上代表专家评分的有序5类目标变量。基于LLM生成特征训练的机器学习模型,其预测性能与科研文本领域的前沿嵌入模型SciBERT相当。该LLM仅使用62个特征,而SciBERT嵌入有768个特征,且这些特征可直接解释,对应文章的方法严谨性、创新性或语法正确性等概念。作为最后一步,我们提取了少量可解释性良好的行动规则。同一LLM特征集在两个主题差异较大的数据集上均取得了持续具有竞争力的结果,表明该方法可跨领域泛化。

英文摘要:

Existing text representations such as embeddings and bag-of-words are not suitable for rule learning due to their high dimensionality and absent or questionable feature-level interpretability. This article explores whether large language models (LLMs) could address this by extracting a small number of interpretable features from text. We demonstrate this process on two datasets (CORD-19 and M17+) containing several thousand scientific articles from multiple disciplines and a target being a proxy for research impact. An evaluation based on testing for the statistically significant correlation with research impact has shown that LLama 2-generated features are semantically meaningful. We consequently used these generated features in text classification to predict the binary target variable representing the citation rate for the CORD-19 dataset and the ordinal 5-class target representing an expert-awarded grade in the M17+ dataset. Machine-learning models trained on the LLM-generated features provided similar predictive performance to the state-of-the-art embedding model SciBERT for scientific text. The LLM used only 62 features compared to 768 features in SciBERT embeddings, and these features were directly interpretable, corresponding to notions such as article methodological rigor, novelty, or grammatical correctness. As the final step, we extract a small number of well-interpretable action rules. Consistently competitive results obtained with the same LLM feature set across both thematically diverse datasets show that this approach generalizes across domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑