arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.10313cs.CL
收藏

大语言模型可以区分社交媒体上的阿片类药物俚语

Large language models can disambiguate opioid slang on social media

Kristy A. Carpenter, Issah A. Samori, Mathew V. Kiang, Keith Humphreys, Anna Lembke, Johannes C. Eichstaedt, Russ B. Altman

AI总结:

大语言模型能有效区分社交媒体上的阿片类药物俚语,提升低发病率主题的数据分析能力。

摘要:

社交媒体文本在监测阿片类药物过量危机趋势方面显示出潜力;然而,社交媒体文本中绝大多数与阿片类药物无关。当利用社交媒体文本来监测正在进行的阿片类药物过量危机趋势时,常用的方法是使用与阿片类药物相关的术语词典作为筛选标准。然而,许多阿片类药物的俚语术语,如“smack”或“blues”,有常见的非阿片类药物含义,使其具有歧义性。大语言模型(LLMs)先进的文本推理能力为大规模区分这些俚语术语提供了机会。我们提出了三个任务,用于评估四种最先进的LLMs(GPT-4、GPT-5、Gemini 2.5 Pro和Claude Sonnet 4.5):一个基于词典的设置,其中LLM必须在给定帖子的上下文中区分特定术语;一个无词典的设置,其中LLM必须在没有词典的情况下根据上下文识别阿片类药物相关的帖子;以及一个新兴俚语的设置,其中LLM必须识别带有模拟新俚语术语的阿片类药物相关的帖子。所有四个LLM在所有任务中都表现出色。在基于词典的子任务中,LLM的F1分数(“fenty”子任务:0.824-0.972;“smack”子任务:0.540-0.862)远超最佳词典策略(0.126和0.009,分别)。在无词典任务中,LLM的F1分数(0.544-0.769)超过了词典(0.080-0.540),并且LLM在召回率上表现一致更高。在新兴俚语任务中,所有LLM的准确率(平均:0.784)、F1分数(平均:0.712)、精确率(平均:0.981)和召回率(平均:0.587)均高于评估的两种词典。我们的结果表明,LLM可以用于识别低发病率主题的相关内容,包括但不限于阿片类药物参考,从而增强提供给下游分析和预测模型的数据。

英文摘要:

Social media text shows promise for monitoring trends in the opioid overdose crisis; however, the overwhelming majority of social media text is unrelated to opioids. When leveraging social media text to monitor trends in the ongoing opioid overdose crisis, a common strategy for identifying relevant content is to use a lexicon of opioid-related terms as inclusion criteria. However, many slang terms for opioids, such as "smack" or "blues," have common non-opioid meanings, making them ambiguous. The advanced textual reasoning capability of large language models (LLMs) presents an opportunity to disambiguate these slang terms at scale. We present three tasks on which to evaluate four state-of-the-art LLMs (GPT-4, GPT-5, Gemini 2.5 Pro, and Claude Sonnet 4.5): a lexicon-based setting, in which the LLM must disambiguate a specific term within the context of a given post; a lexicon-free setting, in which the LLM must identify opioid-related posts from context without a lexicon; and an emergent slang setting, in which the LLM must identify opioid-related posts with simulated new slang terms. All four LLMs showed excellent performance across all tasks. In both subtasks of the lexicon-based setting, LLM F1 scores ("fenty" subtask: 0.824-0.972; "smack" subtask: 0.540-0.862) far exceeded those of the best lexicon strategy (0.126 and 0.009, respectively). In the lexicon-free task, LLM F1 scores (0.544-0.769) surpassed those of lexicons (0.080-0.540), and LLMs demonstrated uniformly higher recall. On emergent slang, all LLMs had higher accuracy (average: 0.784), F1 score (average: 0.712), precision (average: 0.981), and recall (average: 0.587) than the two lexicons assessed. Our results show that LLMs can be used to identify relevant content for low-prevalence topics, including but not limited to opioid references, enhancing data provided to downstream analyses and predictive models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑