arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.17481cs.AI
收藏

格栅:用于对话代理的生成护栏

Lattice: Generative Guardrails for Conversational Agents

Emily Broadhurst, Tawab Safi, Joseph Edell, Vashisht Ganesh, Karime Maamari

AI总结:

Lattice通过迭代优化自构建并持续改进对话代理的护栏,有效提升有害输出防护性能。

摘要:

对话AI系统需要护栏来防止有害输出,但现有方法使用静态规则,无法适应新威胁或部署环境。我们引入了Lattice,一个用于自构建和持续改进护栏的框架。Lattice分两个阶段运作:构建阶段通过迭代模拟和优化从标记示例中构建初始护栏;持续改进阶段通过风险评估、对抗测试和整合自动适应部署的护栏。在ProsocialDialog数据集上评估,Lattice在保留数据上实现了91%的F1分数,优于关键词基线43个百分点,优于LlamaGuard25个百分点,优于NeMo4个百分点。持续改进阶段通过闭环优化在跨领域数据上实现了7个百分点的F1提升。我们的框架表明,有效的护栏可以通过迭代优化自构建。

英文摘要:

Conversational AI systems require guardrails to prevent harmful outputs, yet existing approaches use static rules that cannot adapt to new threats or deployment contexts. We introduce Lattice, a framework for self-constructing and continuously improving guardrails. Lattice operates in two stages: construction builds initial guardrails from labeled examples through iterative simulation and optimization; continuous improvement autonomously adapts deployed guardrails through risk assessment, adversarial testing, and consolidation. Evaluated on the ProsocialDialog dataset, Lattice achieves 91% F1 on held-out data, outperforming keyword baselines by 43pp, LlamaGuard by 25pp, and NeMo by 4pp. The continuous improvement stage achieves 7pp F1 improvement on cross-domain data through closed-loop optimization. Our framework shows that effective guardrails can be self-constructed through iterative optimization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑