基于开源LLM的自信意识细粒度辩论用于心理健康和在线安全的自动化数据增强
Automated Data Enrichment using Confidence-Aware Fine-Grained Debate among Open-Source LLMs for Mental Health and Online Safety
AI总结：
本研究提出CFD框架，利用开源LLM的细粒度辩论实现心理健康和在线安全领域的自动化数据增强，通过实验验证其在多标签增强任务中的优越性能。
摘要：
现实世界的指标在许多自然语言处理（NLP）应用中发挥着重要作用，例如心理健康分析中的生活事件和在线安全中的高风险行为，但由于其动态性质，在训练数据集中标记此类信息往往成本高昂或困难。大型语言模型（LLMs）显示了自动标注的潜力，但多标签预测仍然具有挑战性。在本工作中，我们提出了一个自信意识细粒度辩论（CFD）框架，该框架通过细粒度信息模拟协作标注，以更好地支持自动多标签增强。我们引入了两个新的专家标注资源：一个心理健康Reddit福祉数据集和一个在线安全Facebook分享风险数据集。实验表明，CFD在与一系列基线方法相比时实现了最稳健的增强性能。我们进一步评估了各种无训练增强结合策略，并证明了LLM增强的指标在下游任务中始终有所改善。通过辩论记录纳入的增强特征产生了最大的收益，在在线安全任务中，其表现优于非增强基线，提高了9.9%。
英文摘要：
Real-world indicators play an important role in many Natural Language Processing (NLP) applications, such as life events for mental health analysis and risky behaviours for online safety, yet labelling such information is often costly and/or difficult due to its multi-label and dynamic nature. Large Language Models (LLMs) show promising potential for automated annotation, but the multi-label setting remains challenging. In this work, we propose a Confidence-Aware Fine-Grained Debate (CFD) framework that simulates human collaborative annotation using fine-grained communication to better support automated multi-label enrichment. We introduce two expert-annotated resources: life-event and symptom annotation for a mental health well-being dataset, and a new online safety sharenting dataset. Experiments show that CFD achieves the most robust enrichment performance across tasks, and that the benefit of fine-grained confidence is influenced by its quality and variability. We further evaluate training-free strategies for incorporating enrichment indicators into downstream tasks and show that automated enrichment consistently improves performance, by up to 9.9 Macro-F1 points, with the most effective integration format depending on how the indicator relates to the downstream objective.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
