arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.00940cs.CLcs.AI
收藏

用于敏感话题自动检测的大型语言模型

Large Language Models for Automatic Detection of Sensitive Topics

University of Canterbury(坎特伯雷大学);University of Auckland(奥克兰大学);Uppsala University(乌普萨拉大学);The University of Auckland(奥克兰大学);University of Aizu(会津大学)

Ruoyu Wen, Stephanie Elena Crowe, Kunal Gupta, Xinyue Li, Mark Billinghurst, Simon Hoermann, Dwain Allan, Alaeddin Nassani, Thammathip Piumsomboon

AI总结:

本研究评估了五种大型语言模型在心理健康领域敏感消息检测中的性能,发现GPT-4o以99.5%准确率和0.99 F1分数表现最佳,证明LLMs可作为高效精准的内容审核辅助工具。

摘要:

敏感信息检测在内容审核中至关重要,以维护安全的在线社区。协助这一传统上由人工完成的过程,可以减轻人类审核员繁重而枯燥的任务负担,使他们能够专注于可能带来潜在风险的被标记内容。快速发展的大型语言模型(LLMs)以其理解和处理自然语言的能力而闻名,因此为支持这一过程提供了一种潜在的解决方案。本研究探讨了五种LLMs在两个在线数据集中的心理健康领域内检测敏感消息的能力,并从准确率、精确率、召回率、F1分数和一致性方面评估其性能。我们的研究结果表明,LLMs有潜力作为一种便捷且精确的检测工具集成到审核工作流程中。表现最佳的模型GPT-4o实现了99.5%的平均准确率和0.99的F1分数。我们讨论了在审核工作流程中使用LLMs的优势和潜在挑战,并建议未来研究应解决利用该技术的伦理考量。

英文摘要:

Sensitive information detection is crucial in content moderation to maintain safe online communities. Assisting in this traditionally manual process could relieve human moderators from overwhelming and tedious tasks, allowing them to focus solely on flagged content that may pose potential risks. Rapidly advancing large language models (LLMs) are known for their capability to understand and process natural language and so present a potential solution to support this process. This study explores the capabilities of five LLMs for detecting sensitive messages in the mental well-being domain within two online datasets and assesses their performance in terms of accuracy, precision, recall, F1 scores, and consistency. Our findings indicate that LLMs have the potential to be integrated into the moderation workflow as a convenient and precise detection tool. The best-performing model, GPT-4o, achieved an average accuracy of 99.5\% and an F1-score of 0.99. We discuss the advantages and potential challenges of using LLMs in the moderation workflow and suggest that future research should address the ethical considerations of utilising this technology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑