有益还是有害？探究大型语言模型在网络诱骗预防中的效能
Helpful or Harmful? Exploring the Efficacy of Large Language Models for Online Grooming Prevention
AI总结：
本文评估了大型语言模型在网络诱骗预防中的效能，发现现有模型存在行为不一致和有害回答风险，并提出了改进建议与提示设计警示。
摘要：
强大的生成式大型语言模型（LLMs）正成为公众中流行的问答工具，并被儿童等弱势群体所使用。随着儿童越来越多地接触这些工具，研究人员必须审视LLMs的安全性，特别是对于可能导致严重后果的应用，如在线儿童安全查询。本文探讨了LLMs在网络诱骗预防中的效能，既包括通过建议生成来识别和避免诱骗，也通过改变提供的上下文和提示特异性来研究提示设计对模型性能的影响。在反映超过6,000次LLM交互的结果中，我们发现没有模型明确适合网络诱骗预防，观察到行为一致性的缺乏，以及产生有害答案的潜在风险，尤其是来自开源模型的。我们概述了模型在哪些方面以及如何不足，提供了改进建议，并识别了以令人不安的方式严重影响模型性能的提示设计，这些发现可用于指导最佳实践使用指南。
英文摘要：
Powerful generative Large Language Models (LLMs) are becoming popular tools amongst the general public as question-answering systems, and are being utilised by vulnerable groups such as children. With children increasingly interacting with these tools, it is imperative for researchers to scrutinise the safety of LLMs, especially for applications that could lead to serious outcomes, such as online child safety queries. In this paper, the efficacy of LLMs for online grooming prevention is explored both for identifying and avoiding grooming through advice generation, and the impact of prompt design on model performance is investigated by varying the provided context and prompt specificity. In results reflecting over 6,000 LLM interactions, we find that no models were clearly appropriate for online grooming prevention, with an observed lack of consistency in behaviours, and potential for harmful answer generation, especially from open-source models. We outline where and how models fall short, providing suggestions for improvement, and identify prompt designs that heavily altered model performance in troubling ways, with findings that can be used to inform best practice usage guides.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
