arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.14723cs.CL
收藏

超越信仰:一种非身份安全条件:低数据LoRA微调中强经验性替代身份框架的非身份条件

Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuning

Xinran Zhang

AI总结:

本文研究了低数据LoRA安全微调中四种监督格式对安全性能的影响,发现非身份条件D在三个模型家族中表现最佳,挑战了身份框架假设。

摘要:

安全监督的写法可能比其显式身份内容更重要。我们研究了基于相同核心安全规则的四种监督格式:宪法规则(A)、信仰式身份框架(B)、与世界观/忏悔身份维持尾部匹配的B匹配信仰条件(C)以及匹配的非身份条件(D)。在三个指令微调模型家族(Llama 3.1 8B、Qwen2.5 7B和Gemma 3 4B)上,我们使用重新协调的双裁判流程结合Bedrock托管的DeepSeek v3.2和Sonnet 4.6评估HarmBench。非身份条件D在三个模型家族的完整320行为HarmBench集上表现最强,达到Llama 74.4%的拒绝率,Gemma 76.9%和Qwen 74.1%。相比之下,信仰式框架(B)在Llama和Gemma上优于纯宪法规则(A),但显著低于D,得出整体描述顺序为$D > B > C \geq A > baseline$。这为身份框架假设的强版本提供了有界经验挑战:显式的信仰式身份语言并非此处最强增益的必要条件。在MMLU和ARC-Challenge上的能力评估显示,各条件之间没有显著的权衡。

英文摘要:

How safety supervision is written may matter more than the explicit identity content it contains. We study low-data LoRA safety fine-tuning with four supervision formats built from the same core safety rules: constitutional rules (A), creed-style identity framing (B), a B-matched creed condition with a worldview/confession identity-maintenance tail (C), and a matched non-identity condition (D). Across three instruction-tuned model families (Llama 3.1 8B, Qwen2.5 7B, and Gemma 3 4B), we evaluate HarmBench using a reconciled dual-judge pipeline combining Bedrock-hosted DeepSeek v3.2 and Sonnet 4.6, with disagreement and boundary cases manually resolved. The non-identity condition D is the strongest group on all three model families on the full 320-behavior HarmBench set, reaching 74.4% refusal on Llama, 76.9% on Gemma, and 74.1% on Qwen. By comparison, creed-style framing (B) improves over plain constitutional rules (A) on Llama and Gemma, but remains substantially below D, yielding an overall descriptive ordering of $D > B > C \geq A > baseline$. This provides a bounded empirical challenge to a strong version of the identity-framing hypothesis: explicit creed-style identity language is not necessary for the strongest gains observed here. Capability evaluations on MMLU and ARC-Challenge show no meaningful trade-off across conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑