arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.02684cs.CL
收藏

HiddenGuard:基于专用表示路由器的细粒度安全生成

HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Router

Institute of Computing Technology, CAS(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);University of California, Merced(加州大学默塞德分校);HKUST(香港科技大学)

Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, Ruibin Yuan, Xueqi Cheng

AI总结:

HiddenGuard 提出基于隐藏状态的 Prism 路由器,在令牌级别实时检测并编辑有害内容,实现细粒度安全生成,在保持信息量的同时 F1 超过 90%。

摘要:

随着大语言模型(LLMs)能力日益强大,确保其安全性以及与人类价值观的一致性仍然是一项关键挑战。理想情况下,LLMs 应在提供信息性回应的同时,避免泄露有害或敏感信息。然而,当前的对齐方法严重依赖拒绝策略,例如训练模型完全拒绝有害提示或应用粗粒度过滤器,这些方法受限于其二元性质。这些方法要么完全拒绝信息访问,要么在缺乏足够细微差别的情况下授予访问权限,从而导致过于谨慎的回应或未能检测到细微的有害内容。例如,LLMs 可能因担心滥用而拒绝提供关于药物的基本公开信息。此外,这些基于拒绝的方法难以处理混合内容场景,并且缺乏适应上下文相关敏感性的能力,这可能导致对良性内容的过度审查。为克服这些挑战,我们提出了 HiddenGuard,这是一个用于 LLMs 细粒度安全生成的新框架。HiddenGuard 集成了 Prism(用于流内审核的表示路由器,rePresentation Router for In-Stream Moderation),它与 LLM 协同工作,通过利用中间隐藏状态实现实时、令牌级别的有害内容检测与编辑。这种细粒度方法支持更具细微差别和上下文感知的审核,使模型能够生成信息性回应,同时选择性地编辑或替换敏感信息,而不是直接拒绝。我们还贡献了一个全面的数据集,其中包含跨多种上下文的潜在有害信息的令牌级细粒度标注。我们的实验表明,HiddenGuard 在检测和编辑有害内容方面取得了超过 90% 的 F1 分数,同时保持了模型回应的整体实用性和信息量。

英文摘要:

As Large Language Models (LLMs) grow increasingly powerful, ensuring their safety and alignment with human values remains a critical challenge. Ideally, LLMs should provide informative responses while avoiding the disclosure of harmful or sensitive information. However, current alignment approaches, which rely heavily on refusal strategies, such as training models to completely reject harmful prompts or applying coarse filters are limited by their binary nature. These methods either fully deny access to information or grant it without sufficient nuance, leading to overly cautious responses or failures to detect subtle harmful content. For example, LLMs may refuse to provide basic, public information about medication due to misuse concerns. Moreover, these refusal-based methods struggle to handle mixed-content scenarios and lack the ability to adapt to context-dependent sensitivities, which can result in over-censorship of benign content. To overcome these challenges, we introduce HiddenGuard, a novel framework for fine-grained, safe generation in LLMs. HiddenGuard incorporates Prism (rePresentation Router for In-Stream Moderation), which operates alongside the LLM to enable real-time, token-level detection and redaction of harmful content by leveraging intermediate hidden states. This fine-grained approach allows for more nuanced, context-aware moderation, enabling the model to generate informative responses while selectively redacting or replacing sensitive information, rather than outright refusal. We also contribute a comprehensive dataset with token-level fine-grained annotations of potentially harmful information across diverse contexts. Our experiments demonstrate that HiddenGuard achieves over 90% in F1 score for detecting and redacting harmful content while preserving the overall utility and informativeness of the model's responses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑