arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.04447cs.CVcs.CRcs.LG
收藏

Attention Shift:引导AI远离不安全内容

Attention Shift: Steering AI Away from Unsafe Content

Indian Institute of Technology, Roorkee(印度鲁尔基理工学院)

Shivank Garg, Manyana Tiwari

AI总结:

针对生成模型易生成不安全有害内容的问题,提出一种无需训练的注意力重加权推理方法,可移除不安全概念,经两类越狱提示下的定性定量评估,性能优于现有消融方法,并分析了结果原因与方法局限。

摘要:

本研究针对当前最先进生成模型中不安全或有害内容的生成问题展开,重点研究限制此类内容生成的方法。我们提出一种无需训练的新方法,在推理阶段通过注意力重加权移除不安全概念,无需额外训练。我们将该方法与现有消融方法对比,在直接越狱提示和对抗性越狱提示两类场景下,采用定性与定量指标评估性能。我们还推测了观测结果的潜在原因,并讨论了内容限制的局限性与更广泛影响。

英文摘要:

This study investigates the generation of unsafe or harmful content in state-of-the-art generative models, focusing on methods for restricting such generations. We introduce a novel training-free approach using attention reweighing to remove unsafe concepts without additional training during inference. We compare our method against existing ablation methods, evaluating the performance on both, direct and adversarial jailbreak prompts, using qualitative and quantitative metrics. We hypothesize potential reasons for the observed results and discuss the limitations and broader implications of content restriction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑