arXiv 2203.133972022-03-25cs.CL

GPT-D：通过人工神经语言模型的故意退化诱导痴呆相关的语言异常

GPT-D: Inducing Dementia-related Linguistic Anomalies by Deliberate Degradation of Artificial Neural Language Models

Institute of Health Informatics, University of Minnesota（明尼苏达大学健康信息学研究所）；Mayo Clinic（妙佑医疗国际（梅奥诊所））；Biomedical and Health Informatics, University of Washington（华盛顿大学生物医学与健康信息学系）；Pharmaceutical Care and Health Systems, University of Minnesota（明尼苏达大学药学保健与卫生系统系）

Changye Li, David Knopman, Weizhe Xu, Trevor Cohen, Serguei Pakhomov

备注：: This paper has been accepted by ACL 2022

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

摘要：

涉及微调大量模型参数的深度学习（DL）技术在区分认知健康个体和阿尔茨海默病（AD）患者产生的语言方面表现出色。然而，问题仍然存在，他们的能力，以推广超出小的参考集，是公开的研究。作为直接拟合模型参数的替代方案，我们提出了一种新的方法，通过该方法，将在一般英语文本上预训练的Transformer DL模型（GPT-2）与其人工退化版本（GPT-D）配对，以计算这两个模型的\textit{perplexities}对认知健康和受损个体的语言的比率。该技术接近于广泛使用的"Cookie盗窃"图片描述任务的文本数据上的最先进性能，并且与已建立的替代方案不同，该技术还可以很好地推广到自发对话。此外，GPT-D生成的文本具有已知与AD相关的特征，表明诱导痴呆相关的语言异常。我们的研究朝着更好地理解生成神经语言模型的内部工作机制、它们产生的语言以及痴呆症对人类语言和语言特征的有害影响之间的关系迈出了一步。

英文摘要：

Deep learning (DL) techniques involving fine-tuning large numbers of model parameters have delivered impressive performance on the task of discriminating between language produced by cognitively healthy individuals, and those with Alzheimer's disease (AD). However, questions remain about their ability to generalize beyond the small reference sets that are publicly available for research. As an alternative to fitting model parameters directly, we propose a novel method by which a Transformer DL model (GPT-2) pre-trained on general English text is paired with an artificially degraded version of itself (GPT-D), to compute the ratio between these two models' \textit{perplexities} on language from cognitively healthy and impaired individuals. This technique approaches state-of-the-art performance on text data from a widely used "Cookie Theft" picture description task, and unlike established alternatives also generalizes well to spontaneous conversations. Furthermore, GPT-D generates text with characteristics known to be associated with AD, demonstrating the induction of dementia-related linguistic anomalies. Our study is a step toward better understanding of the relationships between the inner workings of generative neural language models, the language that they produce, and the deleterious effects of dementia on human speech and language characteristics.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。