arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.01604cs.AI
收藏

CRaFT:基于跨层转码器的电路引导拒绝特征选择

CRaFT: Circuit-Guided Refusal Feature Selection via Cross-Layer Transcoders

Yonsei University(延世大学)

Su-Hyeon Kim, Hyundong Jin, Yejin Lee, Yo-Sub Han

AI总结:

提出CRaFT框架,利用跨层转码器构建稀疏特征电路图,通过量化特征间影响及其对最终输出的贡献,选择控制拒绝行为的关键特征,显著提升越狱攻击性能。

摘要:

虽然现代LLM经过对齐以拒绝有害请求,但理解这种拒绝行为背后的机制基础对于模型安全分析至关重要。例如,基于引导的越狱攻击通过识别和操纵稀疏的、类似神经元的拒绝特征来绕过安全护栏。当前的特征选择方法主要依赖于特征在有害提示上的激活强度。然而,仅凭激活强度往往捕捉到主题或词汇线索等表面启发式,而非真正的因果机制。因此,选择拒绝特征需要测量特征间的关系,而不是将每个特征视为孤立的激活信号。基于这一见解,我们提出CRaFT,一个电路引导的框架,用于识别直接控制拒绝决策的关键拒绝特征。CRaFT利用跨层转码器将模型的内部计算映射到稀疏特征电路图中,其中边量化特征间的影响及其对最终输出logits的贡献。通过聚合沿拒绝路径传播的效应,CRaFT有效地对最具影响力的特征进行排序。在四个越狱基准上的广泛评估表明,与当前最先进方法相比,CRaFT将平均性能从6.7%提高到57.4%,并生成更具体的有害补全。

英文摘要:

While modern LLMs are aligned to refuse harmful requests, it is essential to understand the underlying mechanistic basis of this refusal behavior for model safety analysis. For example, steering-based jailbreak attacks exploit this by identifying and manipulating sparse, neuron-like refusal features to bypass safety guardrails. Current feature selection methods primarily rely on how strongly features activate on harmful prompts. However, activation strength alone often captures superficial heuristics such as topic or lexical cues, rather than the true causal mechanisms. Thus, selecting refusal features requires measuring inter-feature relationships, rather than treating each feature as an isolated activation signal. Based on this insight, we propose CRaFT, a circuit-guided framework for identifying critical refusal features that directly govern the refusal decision. CRaFT leverages cross-layer transcoders to map the model's internal computations into a sparse feature circuit graph, where edges quantify inter-feature influences and their contributions to the final output logits. By aggregating the effects propagating along the paths to refusal, CRaFT effectively ranks the most influential features. Extensive evaluations across four jailbreak benchmarks show that CRaFT significantly improves average performance from 6.7% to 57.4% and generates more specific harmful completions compared to current SOTA methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑