基于MentalRoBERTa的分层双头模型用于自杀风险评估
Hierarchical Dual-Head Model for Suicide Risk Assessment via MentalRoBERTa
AI总结:
针对社交媒体自杀风险检测中的类别不平衡、时间复杂性和风险级别双重性质,提出基于MentalRoBERTa的分层双头模型,结合CORAL和分类头及组合损失,实现四类风险分类,以宏F1评估。
摘要:
社交媒体平台已成为识别自杀风险的重要来源,但自动化检测系统面临多重挑战,包括严重的类别不平衡、发帖模式的时间复杂性以及风险水平既具有序性又具有类别性的双重性质。本文提出一种基于MentalRoBERTa的分层双头神经网络,用于将自杀风险分类为四个级别:指标、意念、行为和尝试。该模型在共享序列表示上采用两个互补的预测头:一个CORAL(一致秩逻辑)头,用于保持风险级别之间的有序关系;以及一个标准分类头,用于实现灵活的类别区分。一个具有8头多头注意力的3层Transformer编码器对跨帖子序列的时间依赖性进行建模,而显式的时间间隔嵌入捕获发帖行为动态。该模型使用组合损失函数(0.5 CORAL + 0.3 交叉熵 + 0.2 焦点损失)进行训练,同时解决有序结构保持、过度自信减少和类别不平衡问题。为了提高计算效率,我们冻结了MentalRoBERTa的前6层(50%)并采用混合精度训练。该模型使用5折分层交叉验证进行评估,以宏F1分数作为主要指标。
英文摘要:
Social media platforms have become important sources for identifying suicide risk, but automated detection systems face multiple challenges including severe class imbalance, temporal complexity in posting patterns, and the dual nature of risk levels as both ordinal and categorical. This paper proposes a hierarchical dual-head neural network based on MentalRoBERTa for suicide risk classification into four levels: indicator, ideation, behavior, and attempt. The model employs two complementary prediction heads operating on a shared sequence representation: a CORAL (Consistent Rank Logits) head that preserves ordinal relationships between risk levels, and a standard classification head that enables flexible categorical distinctions. A 3-layer Transformer encoder with 8-head multi-head attention models temporal dependencies across post sequences, while explicit time interval embeddings capture posting behavior dynamics. The model is trained with a combined loss function (0.5 CORAL + 0.3 Cross-Entropy + 0.2 Focal Loss) that simultaneously addresses ordinal structure preservation, overconfidence reduction, and class imbalance. To improve computational efficiency, we freeze the first 6 layers (50%) of MentalRoBERTa and employ mixed-precision training. The model is evaluated using 5-fold stratified cross-validation with macro F1 score as the primary metric.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。