层次化正交残差扩展用于大语言模型中的精确大规模编辑
Hierarchical Orthogonal Residual Spread for Precise Massive Editing in Large Language Models
AI总结：
本文提出HORSE方法，通过层次化正交残差扩展实现大语言模型中的精确大规模编辑，通过理论分析和实验验证其有效性。
摘要：
大语言模型（LLMs）在各种领域展现出卓越的性能，但它们面临关键的安全问题。模型编辑已成为缓解这些问题的有效方法。现有的模型编辑方法通常专注于优化一个融合新旧知识的信息矩阵。虽然有效，但这些方法可能计算成本高且可能导致冲突。相反，我们关注信息矩阵的层次化正交残差扩展（HORSE），这能减少噪声梯度并从不同角度实现更稳定的编辑。我们通过与几种流行方法的清晰理论比较以及在两个数据集上多个LLMs上的广泛实验，展示了HORSE方法的有效性。结果表明，HORSE在多样化场景中保持了精确的大规模编辑。代码可在https://github.com/XiaojieGu/HORSE上获得。
英文摘要：
Large language models (LLMs) exhibit exceptional performance across various domains, yet they face critical safety concerns. Model editing has emerged as an effective approach to mitigate these issues. Existing model editing methods often focus on optimizing an information matrix that blends new and old knowledge. While effective, these approaches can be computationally expensive and may cause conflicts. In contrast, we shift our attention to Hierarchical Orthogonal Residual SprEad of the information matrix, which reduces noisy gradients and enables more stable edits from a different perspective. We demonstrate the effectiveness of our method HORSE through a clear theoretical comparison with several popular methods and extensive experiments conducted on two datasets across multiple LLMs. The results show that HORSE maintains precise massive editing across diverse scenarios. The code is available at https://github.com/XiaojieGu/HORSE
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
