在线交流中心理健康污名的自动评估
Automatic Evaluation of Mental Health Stigma in Online Communication
浏览论文内容
中文总结 AI 辅助
本研究提出一个基于理论的心理健康污名自动评估基准，涵盖细粒度分类法，并评估大型语言模型，发现现有模型难以准确捕获污名，LLM需明确规则以避免过度预测。
中文摘要 AI 辅助
心理健康污名具有极其有害的影响，但其复杂性使其难以评估。污名可能涉及明确的贬损，但也包括更微妙的责备、恐惧、家长式怜悯、社会距离、结构性排斥和歧视形式。我们引入了一个基于理论的心理健康污名自动评估基准，该基准用于在线交流，包含自然发生的在线新闻和社交媒体文本，这些文本根据多种心理健康状况的污名细粒度分类法进行了标注。我们的标注框架包括一个二元污名检测任务和一个多层次分类法，涵盖（i）污名模式，（ii）领域，以及（iii）特定污名形式的特定组成部分。我们将此框架应用于提及六种心理健康状况的文本，并评估大型语言模型以及用于检测情感、有毒性和仇恨言论的污名相关分类器。结果表明，心理健康污名并不能被训练用于检测这些邻近构念的模型很好地捕获，而大型语言模型往往过度预测污名，除非给出明确的操作规则——这反映了决策规则在人类标注中的重要性。我们发布了基准的公开部分、标注、污名的原型示例和代码，网址为：此 https URL。
英文摘要
Mental health stigma has profoundly harmful impacts but its complexity makes it difficult to evaluate. Stigma may involve explicit derogation, but also subtler forms of blame, fear, paternalistic pity, social distancing, structural exclusion, and discrimination. We introduce a theory-grounded benchmark for automatic evaluation of mental health stigma in online communication, consisting of naturally occurring online news and social media text annotated with a fine-grained taxonomy of stigma across multiple mental health conditions. Our annotation framework comprises a binary stigma-detection task and a multi-level taxonomy covering (i) stigma mode, (ii) domain, and (iii) specific components of certain forms of stigma. We apply this framework to texts mentioning six mental health conditions and evaluate large language models alongside stigma-related classifiers for detecting sentiment, toxicity, and hate speech. Results show that mental health stigma is not well captured by models trained to detect these neighboring constructs, and that LLMs often overpredict stigma unless given explicit operational rules - mirroring the importance of decision rules in human annotation. We release the publicly available part of benchmark, annotations, prototypical exemplars of stigma and code at: https://github.com/jemimakang/mh_stigma.
发表机构
- The University of Melbourne（墨尔本大学）
- University of Tübingen（图宾根大学）
机构由 AI 辅助整理，请以论文原文为准。
