arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.17880cs.CLcs.AI
收藏

恶意向量的跨语言迁移具有方向不对称性

Cross-Lingual Transfer of Evil Vectors Is Directionally Asymmetric

University of Southern California(南加州大学);KAIST(韩国科学技术院)

Juann Kim, Minku Kang

AI总结:

本研究发现恶意向量的跨语言迁移具有方向不对称性,高资源语言提取的恶意向量可有效迁移至低资源语言且保留通用知识,无需额外单语言数据即可引导低资源语言模型行为。

摘要:

角色向量是大型语言模型(LLM)残差流激活中的线性方向,可编码行为特征并实现对模型行为的细粒度控制,其中恶意角色向量与理解有害行为、越狱漏洞及有害知识的遗忘尤为相关。然而,该向量是否反映真正的跨语言概念,而非其提取语言的人工产物,尚未得到验证。我们在四种开源指令调优LLM上,针对两种高资源语言(英语、西班牙语)和两种低资源语言(斯瓦希里语、泰卢固语)开展系统的恶意向量迁移研究,发现迁移具有方向不对称性:低资源语言中难以可靠获取恶意向量,且所得向量即使在自身语言内也基本无法用于引导;相比之下,从高资源语言提取的向量可有效迁移至低资源语言输入,同时通过MMLU(大规模多任务语言理解)衡量,能在很大程度上保留通用知识。几何分析进一步支持该模式:两种高资源语言的恶意向量紧密对齐,而低资源语言的向量与该共享方向仅弱对齐。这些结果表明,存在共享的跨语言恶意方向,一旦从高资源语言可靠提取,即可重复用于引导低资源语言的行为,无需额外的单语言数据收集。

英文摘要:

Persona vectors are linear directions in the residual-stream activations of LLMs that encode behavioral traits and enable fine-grained control over model behavior. Among them, the evil persona vector is particularly relevant to understanding harmful behavior, jailbreak vulnerabilities, and the unlearning of harmful knowledge. Yet whether this vector reflects a genuinely cross-lingual concept, rather than an artifact of its extraction language, remains untested. We conduct a systematic evil-vector transfer study across two high-resource languages (English, Spanish) and two low-resource languages (Swahili, Telugu) on four open-source instruction-tuned LLMs, and find that transfer is directionally asymmetric. Evil vectors are harder to obtain reliably from low-resource languages, and the resulting vectors are largely ineffective at steering, even within their own language. In contrast, vectors extracted from high-resource languages transfer effectively to low-resource inputs while largely preserving general knowledge, as measured by MMLU. Geometric analysis further supports this pattern: evil vectors from the two high-resource languages are closely aligned, whereas low-resource vectors align only weakly with this shared direction. These results suggest that a shared cross-lingual evil direction exists and, once reliably extracted from a high-resource language, can be reused to steer behavior in low-resource languages without additional per-language data collection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑