Kelp:通过潜在动态引导的风险检测实现大型模型的流式防护
Kelp: A Streaming Safeguard for Large Models via Latent Dynamics-Guided Risk Detection
AI总结:
提出Kelp插件框架,利用流式潜在动态头和ATC损失在LM生成流程内实现实时风险检测,以极低延迟和参数量显著超越现有事后防护方法。
摘要:
大型模型(LMs)是强大的内容生成器,但其开放性也可能引入潜在风险,例如生成有害或带有偏见的内容。现有防护措施大多执行事后检测,可能在被捕获之前就暴露出不安全内容,而延迟限制进一步促使它们采用轻量级模型,从而限制了检测准确性。在这项工作中,我们提出了Kelp,一个新颖的插件框架,能够在LM生成流程内实现流式风险检测。Kelp通过流式潜在动态头(SLD)利用LM的中间隐藏状态,对生成序列中风险的时序演变进行建模,以实现更准确的实时风险检测。为了确保在实际应用中实现可靠的流式审核,我们引入了锚定时序一致性(ATC)损失,通过嵌入先良性后有害的时序先验来强制执行单调的有害性预测。此外,为了对流式防护措施进行严格评估,我们还提出了StreamGuardBench——一个基于模型的基准测试,具有来自每个受保护模型的即时响应,反映了文本和视觉语言任务中的真实流式场景。在多种模型和数据集上,Kelp始终优于最先进的事后防护措施和先前的插件探测方法(平均F1高出15.61%),同时仅使用20M参数,且每词元增加的延迟不到0.5毫秒。
英文摘要:
Large models (LMs) are powerful content generators, yet their open-ended nature can also introduce potential risks, such as generating harmful or biased content. Existing guardrails mostly perform post-hoc detection that may expose unsafe content before it is caught, and the latency constraints further push them toward lightweight models, limiting detection accuracy. In this work, we propose Kelp, a novel plug-in framework that enables streaming risk detection within the LM generation pipeline. Kelp leverages intermediate LM hidden states through a Streaming Latent Dynamics Head (SLD), which models the temporal evolution of risk across the generated sequence for more accurate real-time risk detection. To ensure reliable streaming moderation in real applications, we introduce an Anchored Temporal Consistency (ATC) loss to enforce monotonic harm predictions by embedding a benign-then-harmful temporal prior. Besides, for a rigorous evaluation of streaming guardrails, we also present StreamGuardBench-a model-grounded benchmark featuring on-the-fly responses from each protected model, reflecting real-world streaming scenarios in both text and vision-language tasks. Across diverse models and datasets, Kelp consistently outperforms state-of-the-art post-hoc guardrails and prior plug-in probes (15.61% higher average F1), while using only 20M parameters and adding less than 0.5 ms of per-token latency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。