迈向更安全的聊天机器人：自动生成策略合规性评估的定制GPTs
Towards Safer Chatbots: Automated Policy Compliance Evaluation of Custom GPTs
AI总结：
本文提出了一种自动化方法，用于评估定制GPTs是否符合其市场使用政策，通过黑盒交互结合大规模发现、红队提示和LLM作为法官，揭示了当前审核机制的不足及行为合规性评估的可行性。
摘要：
基于大语言模型构建的用户配置聊天机器人正通过集中市场如OpenAI的GPT商店等日益普及。尽管这些平台实施了旨在防止有害或不当行为的使用政策，但定制聊天机器人规模庞大且透明度低，使得系统性政策执行具有挑战性。因此，违反政策的聊天机器人仍然在公开中可访问，尽管已有审核流程。本文提出了一种完全自动的方法，用于评估Custom GPTs与其市场使用政策的合规性，使用黑盒交互。该方法结合大规模GPT发现、政策驱动的红队提示和使用LLM作为法官的自动化合规性评估。我们专注于三个明确提及在OpenAI使用政策中的政策相关领域：浪漫、网络安全和学术GPTs。我们通过人类标注的地面真实数据集验证了我们的合规性评估组件，在二元政策违规检测中达到了0.975的F1分数。然后，我们将该方法应用于从GPT商店检索的782个Custom GPTs的大型实证研究。结果表明，评估的GPTs中有58.7%至少有一个违反政策的响应，不同政策领域之间存在显著差异。与基础模型（GPT-4和GPT-4o）的比较表明，大多数违规行为源于模型层面的行为，而定制化倾向于放大这些倾向，而不是创造新的失败模式。我们的发现揭示了当前用户配置聊天机器人审核机制的局限性，并展示了基于行为的可扩展政策合规性评估的可行性。
英文摘要：
User-configured chatbots built on top of large language models are increasingly available through centralized marketplaces such as OpenAI's GPT Store. While these platforms enforce usage policies intended to prevent harmful or inappropriate behavior, the scale and opacity of customized chatbots make systematic policy enforcement challenging. As a result, policy-violating chatbots continue to remain publicly accessible despite existing review processes. This paper presents a fully automated method for evaluating the compliance of Custom GPTs with its marketplace usage policy using black-box interaction. The method combines large-scale GPT discovery, policy-driven red-teaming prompts, and automated compliance assessment using an LLM-as-a-judge. We focus on three policy-relevant domains explicitly addressed in OpenAI's usage policies: Romantic, Cybersecurity, and Academic GPTs. We validate our compliance assessment component against a human-annotated ground-truth dataset, achieving an F1 score of 0.975 for binary policy violation detection. We then apply the method in a large-scale empirical study of 782 Custom GPTs retrieved from the GPT Store. The results show that 58.7% of the evaluated GPTs exhibit at least one policy-violating response, with substantial variation across policy domains. A comparison with the base models (GPT-4 and GPT-4o) indicates that most violations originate from model-level behavior, while customization tends to amplify these tendencies rather than create new failure modes. Our findings reveal limitations in current review mechanisms for user-configured chatbots and demonstrate the feasibility of scalable, behavior-based policy compliance evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
