arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.11575cs.CLcs.AI
收藏

大语言模型中的概念吸引子及其应用

Concept Attractors in LLMs and their Applications

University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Sotirios Panagiotis Chytas, Vikas Singh

AI总结:

本文提出基于吸引子的简单方法,用于大语言模型中的概念映射和实际任务处理,提供高效替代方案。

摘要:

大语言模型(LLMs)常常将语义相关提示映射到特定层的相似内部表示,即使它们的表层形式差异很大。我们展示了这种行为可以通过迭代函数系统(IFS)来解释,其中层充当指向特定概念吸引子的收缩映射。我们利用这一见解,开发了简单且无需训练的方法,直接在这些吸引子上操作,以解决广泛的实际任务,包括语言翻译、减少幻觉、设置护栏以及合成数据生成。尽管这些基于吸引子的干预方法简单,但它们在匹配或超越专门基线方面表现优异,为重力微调提供了一个高效的替代方案,在基线表现不佳的场景中具有泛化能力。

英文摘要:

Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these Attractors to solve a wide range of practical tasks, including language translation, hallucination reduction, guardrailing, and synthetic data generation. Despite their simplicity, these Attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑