GAF-Guard：面向大型语言模型风险管理和治理的智能体框架
GAF-Guard: An Agentic Framework for Risk Management and Governance in Large Language Models
AI总结：
GAF-Guard提出以用户、用例和模型为中心的智能体框架，通过自主智能体识别风险、激活检测工具并持续监控，以增强LLM应用的安全性和治理。
摘要：
随着大型语言模型（LLM）在各领域的应用日益广泛，其大规模采用需要严格的监控，以防止意外负面后果并确保鲁棒性。此外，LLM的设计必须与人类价值观保持一致，例如防止有害内容并确保负责任的使用。当前用于监控生产环境中LLM的自动化系统和解决方案主要集中于LLM特有的问题，如幻觉等，很少考虑特定用例和用户偏好的需求。本文介绍了GAF-Guard，一种新颖的用于LLM治理的智能体框架，该框架将用户、用例和模型本身置于中心位置。该框架旨在检测和监控基于LLM的应用部署相关的风险。该方法对自主智能体进行建模，这些智能体在特定用例中识别风险、激活风险检测工具，并促进持续监控和报告，以增强AI安全性和用户期望。代码可在https://github.com/IBM/risk-atlas-nexus-demos/tree/main/gaf-guard获取。
英文摘要：
As Large Language Models (LLMs) continue to be increasingly applied across various domains, their widespread adoption necessitates rigorous monitoring to prevent unintended negative consequences and ensure robustness. Furthermore, LLMs must be designed to align with human values, like preventing harmful content and ensuring responsible usage. The current automated systems and solutions for monitoring LLMs in production are primarily centered on LLM-specific concerns like hallucination etc, with little consideration given to the requirements of specific use-cases and user preferences. This paper introduces GAF-Guard, a novel agentic framework for LLM governance that places the user, the use-case, and the model itself at the center. The framework is designed to detect and monitor risks associated with the deployment of LLM based applications. The approach models autonomous agents that identify risks, activate risk detection tools, within specific use-cases and facilitate continuous monitoring and reporting to enhance AI safety, and user expectations. The code is available at https://github.com/IBM/risk-atlas-nexus-demos/tree/main/gaf-guard.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
