迈向基于大型语言模型的可解释心理健康分析
Towards Interpretable Mental Health Analysis with Large Language Models
AI总结：
本文在 5 项任务、11 个数据集上评估 LLM 的心理健康分析能力，结合情绪提示与少样本策略生成决策解释，并通过人工评估和自动指标基准验证 ChatGPT 的可解释性潜力。
摘要：
诸如 ChatGPT 之类的最新大型语言模型（LLM）在自动化心理健康分析方面展现出强大能力。然而，现有相关研究存在若干局限，包括评估不充分、缺乏提示策略，以及忽视探索 LLM 的可解释性。为弥补这些差距，我们在跨越 5 项任务的 11 个数据集上全面评估了 LLM 的心理健康分析与情绪推理能力。我们利用无监督和远程监督的情绪信息，探索了不同提示策略的效果。基于这些提示，我们通过指示 LLM 为其每项决策生成解释，探索将其用于可解释心理健康分析。我们开展严格的人工评估以衡量所生成解释的质量，由此构建了一个包含 163 条经人工评估解释的新数据集。我们在该数据集上对现有自动评估指标进行基准测试，以指导未来相关工作。结果表明，ChatGPT 展现出较强的上下文学习能力，但与先进的任务专用方法之间仍存在显著差距。借助情绪线索和专家撰写的少样本示例进行细致的提示工程，也能有效提升心理健康分析性能。此外，ChatGPT 生成的解释接近人类表现，显示出其在可解释心理健康分析中的巨大潜力。
英文摘要：
The latest large language models (LLMs) such as ChatGPT, exhibit strong capabilities in automated mental health analysis. However, existing relevant studies bear several limitations, including inadequate evaluations, lack of prompting strategies, and ignorance of exploring LLMs for explainability. To bridge these gaps, we comprehensively evaluate the mental health analysis and emotional reasoning ability of LLMs on 11 datasets across 5 tasks. We explore the effects of different prompting strategies with unsupervised and distantly supervised emotional information. Based on these prompts, we explore LLMs for interpretable mental health analysis by instructing them to generate explanations for each of their decisions. We convey strict human evaluations to assess the quality of the generated explanations, leading to a novel dataset with 163 human-assessed explanations. We benchmark existing automatic evaluation metrics on this dataset to guide future related works. According to the results, ChatGPT shows strong in-context learning ability but still has a significant gap with advanced task-specific methods. Careful prompt engineering with emotional cues and expert-written few-shot examples can also effectively improve performance on mental health analysis. In addition, ChatGPT generates explanations that approach human performance, showing its great potential in explainable mental health analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
