重新思考自杀意念检测：一个可信的标注框架与跨语言模型评估
Rethinking Suicidal Ideation Detection: A Trustworthy Annotation Framework and Cross-Lingual Model Evaluation
AI总结：
本文针对自杀意念检测中语言覆盖有限和标注不可靠的问题，构建土耳其语语料库并提出融合人工与大语言模型的可信标注框架，通过跨语言迁移学习评估标签可靠性与模型一致性，揭示零样本迁移的局限并倡导心理健康NLP的透明与可靠性。
摘要：
自杀意念检测对于实时自杀预防至关重要，然而其进展面临两个尚未被充分探索的挑战：语言覆盖范围有限和标注实践不可靠。大多数可用数据集都是英文的，但即使在这些数据集中，高质量的人工标注数据仍然稀缺。因此，许多研究依赖现成的预标注数据集，而不检查其标注过程或标签可靠性。其他语言数据集的缺乏进一步限制了通过人工智能（AI）实现全球自杀预防的可能性。在本研究中，我们通过构建一个源自社交媒体帖子的新型土耳其语自杀意念语料库，并引入一个涉及三名人工标注者和两个大语言模型（LLMs）的资源高效标注框架，来解决其中一个空白。随后，我们通过在该数据集和三个流行的英文自杀意念检测数据集上对标签可靠性和模型一致性进行双向评估，并使用八个预训练情感和情绪分类器进行迁移学习，来解决其余空白。这些transformer有助于评估标注一致性，并以人工标注数据为基准对模型性能进行基准测试。我们的研究结果强调，在心理健康自然语言处理（NLP）中需要更严格、更具语言包容性的标注和评估方法，同时证明了流行模型在零样本迁移学习下的表现值得怀疑。我们倡导在心理健康NLP中提高模型训练和数据集构建的透明度，优先考虑数据和模型的可靠性。
英文摘要：
Suicidal ideation detection is critical for real-time suicide prevention, yet its progress faces two under-explored challenges: limited language coverage and unreliable annotation practices. Most available datasets are in English, but even among these, high-quality, human-annotated data remains scarce. As a result, many studies rely on available pre-labeled datasets without examining their annotation process or label reliability. The lack of datasets in other languages further limits the global realization of suicide prevention via artificial intelligence (AI). In this study, we address one of these gaps by constructing a novel Turkish suicidal ideation corpus derived from social media posts and introducing a resource-efficient annotation framework involving three human annotators and two large language models (LLMs). We then address the remaining gaps by performing a bidirectional evaluation of label reliability and model consistency across this dataset and three popular English suicidal ideation detection datasets, using transfer learning through eight pre-trained sentiment and emotion classifiers. These transformers help assess annotation consistency and benchmark model performance against manually labeled data. Our findings underscore the need for more rigorous, language-inclusive approaches to annotation and evaluation in mental health natural language processing (NLP) while demonstrating the questionable performance of popular models with zero-shot transfer learning. We advocate for transparency in model training and dataset construction in mental health NLP, prioritizing data and model reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
