SteeringSafety：针对大语言模型在多安全视角下的表征引导基准测试
SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives
AI总结：
本研究推出SteeringSafety基准，测试DIM等引导方法在3款大模型的9种安全视角表现，发现方法与模型、视角的匹配影响性能，且存在多视角纠缠问题，需多安全角度评估引导方法。
摘要：
我们推出了SteeringSafety，这是一个涵盖9种安全视角、涉及18个数据集的基准，用于评估表征引导方法。现有研究虽强调表征引导的通用能力，但本研究聚焦于拒绝、偏见、幻觉、社会行为、推理、认知完整性和规范判断等安全视角。SteeringSafety为最先进的引导方法提供模块化构建块，可统一实现DIM、ACE、CAA、PCA和LAT，并支持条件引导等最新增强功能。在Gemma-2-2B、Llama-3.1-8B和Qwen-2.5-7B上的结果显示，良好的引导性能取决于方法、模型与特定视角的匹配。例如，DIM始终有效，但所有方法均存在显著的纠缠问题，即提升某一安全视角的效果往往会显著改变其他视角的性能。社会行为最易受影响（退化幅度达76%），拒绝引导（越狱）常损害常识道德等规范判断（幅度达26%），幻觉引导会使不同模型的政治立场发生不可预测的转变，从向右25%到向左28%不等。这些发现表明，需从多个安全角度而非单一目标行为来理解引导方法。
英文摘要：
We introduce SteeringSafety, a benchmark for evaluating representation steering methods across nine safety perspectives spanning 18 datasets. While prior work highlights the general capabilities of representation steering, we focus on safety perspectives including refusal, bias, hallucination, social behaviors, reasoning, epistemic integrity, and normative judgment. SteeringSafety provides modularized building blocks for state-of-the-art steering methods, enabling unified implementation of DIM, ACE, CAA, PCA, and LAT with recent enhancements such as conditional steering. Results on Gemma-2-2B, Llama-3.1-8B, and Qwen-2.5-7B show that strong steering performance depends on the pairing of method, model, and specific perspective. For instance, DIM is consistently effective, yet all methods exhibit substantial entanglement, where improving effectiveness on one safety perspective often significantly changes performance on others. Social behaviors are most vulnerable (degradation up to 76%), refusal steering (jailbreaking) frequently compromises normative judgment such as commonsense morality (up to 26%), and hallucination steering shifts political views unpredictably across models, ranging from a 25% shift to the right to a 28% shift to the left. These findings show the need to understand steering methods through multiple safety angles rather than a single target behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
