你的语言模型秘密包含个性子网络
Your Language Model Secretly Contains Personality Subnetworks
AI总结：
本研究发现大型语言模型内部已嵌入身份子网络，无需外部知识即可实现身份切换和行为调整。
摘要：
人类根据社会情境切换不同的身份。大型语言模型（LLMs）在采用不同身份和行为方面表现出相似的灵活性。然而，现有方法通常通过外部知识如提示、检索增强生成（RAG）或微调来适应此类行为。我们问：LLMs是否真的需要外部上下文或参数来适应不同行为，还是它们已经在其参数空间中包含此类知识？在本工作中，我们证明LLMs已经在其参数空间中包含针对不同身份的子网络。使用小型校准数据集，我们识别出与不同身份相关的不同激活特征。受这些统计数据的指导，我们开发了一种掩码策略，以隔离轻量级的身份子网络。基于这些发现，我们进一步讨论：如何从模型中发现导致二元对立身份（如内向-外向）的子网络？为了进一步增强二元对立场景中的分离，我们引入了一种对比修剪策略，以识别负责对立身份之间统计差异的参数。我们的方法完全无需训练，仅依赖语言模型已有的参数空间。在多样化的评估设置中，所得到的子网络在身份对齐方面显著优于需要外部知识的基线方法，同时更加高效。我们的发现表明，多样化的人类行为并非仅仅在LLMs中被诱导，而是已经嵌入其参数空间中，指向了大型语言模型可控和可解释个性化的新视角。
英文摘要：
Humans shift between different personas depending on social context. Large Language Models (LLMs) demonstrate a similar flexibility in adopting different personas and behaviors. Existing approaches, however, typically adapt such behavior through external knowledge such as prompting, retrieval-augmented generation (RAG), or fine-tuning. We ask: do LLMs really need external context or parameters to adapt to different behaviors, or do they already have such knowledge embedded in their parameters? In this work, we show that LLMs already contain persona-specialized subnetworks in their parameter space. Using small calibration datasets, we identify distinct activation signatures associated with different personas. Guided by these statistics, we develop a masking strategy that isolates lightweight persona subnetworks. Building on the findings, we further discuss: how can we discover opposing subnetwork from the model that lead to binary-opposing personas, such as introvert-extrovert? To further enhance separation in binary opposition scenarios, we introduce a contrastive pruning strategy that identifies parameters responsible for the statistical divergence between opposing personas. Our method is entirely training-free and relies solely on the language model's existing parameter space. Across diverse evaluation settings, the resulting subnetworks exhibit significantly stronger persona alignment than baselines that require external knowledge while being more efficient. Our findings suggest that diverse human-like behaviors are not merely induced in LLMs, but are already embedded in their parameter space, pointing toward a new perspective on controllable and interpretable personalization in large language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
