大语言模型中通过神经元级编辑的可控价值对齐
Controllable Value Alignment in Large Language Models through Neuron-Level Editing
AI总结：
提出NeVA框架，通过识别稀疏价值相关神经元并进行推理时激活编辑，实现细粒度可控价值对齐，减少价值泄漏并保持通用能力。
摘要：
随着大语言模型对人类行为和决策的影响不断扩大，使其与人类价值观对齐变得越来越重要。然而，现有的基于引导的对齐方法存在可控性有限的问题：引导目标价值往往会无意中激活其他非目标价值。为了描述这一局限性，我们引入了价值泄漏这一诊断概念，它捕捉了价值引导过程中非目标价值的非预期激活，并基于Schwartz价值理论提出了归一化泄漏度量。基于此分析，我们提出了NeVA，一种用于大语言模型中可控价值对齐的神经元级编辑框架。NeVA识别稀疏的、与价值相关的神经元，并进行推理时激活编辑，无需参数更新或重新训练即可实现细粒度控制。实验表明，NeVA在实现更强的目标价值对齐的同时，对通用能力的性能下降更小。此外，NeVA显著降低了平均泄漏，残余效应主要局限于语义相关的价值类别。总体而言，NeVA为价值对齐提供了一种更可控且可解释的机制。
英文摘要：
Aligning large language models (LLMs) with human values has become increasingly important as their influence on human behavior and decision-making expands. However, existing steering-based alignment methods suffer from limited controllability: steering a target value often unintentionally activates other, non-target values. To characterize this limitation, we introduce value leakage, a diagnostic notion that captures the unintended activation of non-target values during value steering, along with a normalized leakage metric grounded in Schwartz's value theory. In light of this analysis, we propose NeVA, a neuron-level editing framework for controllable value alignment in LLMs. NeVA identifies sparse, value-relevant neurons and performs inference-time activation editing, enabling fine-grained control without parameter updates or retraining. Experiments show that NeVA achieves stronger target value alignment while incurring smaller performance degradation on general capability. Moreover, NeVA significantly reduces the average leakage, with residual effects largely confined to semantically related value classes. Overall, NeVA offers a more controllable and interpretable mechanism for value alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
