arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.20031cs.AIcs.LG
收藏

潜在自我反思:模型可以检测先前概念注入

Latent Introspection: Models Can Detect Prior Concept Injections

ACS Research, CTS, Charles University(ACS研究机构、CTs、查尔斯大学)

Theia Pearson-Vogel, Martin Vanek, Raymond Douglas, Jan Kulveit

AI总结:

Qwen 32B模型能检测并识别先前注入的概念,通过提供准确的AI自我反思机制信息可显著提升检测效果,同时提高注入概念间的互信息。

摘要:

我们揭示了一个Qwen 32B模型中潜在的自我反思能力,证明该模型能够检测到概念已被注入到其早期上下文中的情况,并能识别出哪个概念被注入。虽然模型在采样输出中否认了注入,但logit lens分析显示残差流中存在清晰的检测信号,这些信号在最终层中被削弱。此外,通过向模型提供关于AI自我反思机制的准确信息,可以显著增强这种效果:仅在假阳性增加0.6%的情况下,对注入的敏感性就从0.3%增加到39.9%。此外,九个注入和恢复的概念之间的互信息从0.61比特增加到1.05比特,排除了通用噪声解释。我们的结果表明,模型具有令人惊讶的自我反思和引导意识能力,这种能力容易被忽视,对潜在推理和安全有影响。

英文摘要:

We uncover a latent capacity for introspection in a Qwen 32B model, demonstrating that the model can detect when concepts have been injected into its earlier context and identify which concept was injected. While the model denies injection in sampled outputs, logit lens analysis reveals clear detection signals in the residual stream, which are attenuated in the final layers. Furthermore, prompting the model with accurate information about AI introspection mechanisms can dramatically strengthen this effect: the sensitivity to injection increases massively (0.3% -> 39.9%) with only a 0.6% increase in false positives. Also, mutual information between nine injected and recovered concepts rises from 0.61 bits to 1.05 bits, ruling out generic noise explanations. Our results demonstrate models can have a surprising capacity for introspection and steering awareness that is easy to overlook, with consequences for latent reasoning and safety.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑