Persona-Pruner: 为角色扮演雕琢轻量级模型
Persona-Pruner: Sculpting Lightweight Models for Role-Playing
AI总结：
提出Persona-Pruner框架，通过从单个描述中隔离特定角色的子网络来剪枝语言模型，在保持角色扮演性能的同时大幅降低计算成本，性能下降比最强基线减少93.8%。
摘要：
语言模型（LMs）作为角色扮演聊天机器人展现出显著潜力，在给定角色或用户画像规范时，能够提供一致且风格化的交互。然而，将这些能力应用于现实世界应用（例如，众多NPC同时交互的生态系统）时，由于过高的计算成本，暴露了关键的效率问题。在本文中，我们质疑将完整的通用模型专用于单一角色的必要性，假设特定角色身份仅依赖于模型总容量的一小部分。我们观察到，朴素地剪枝LM通常会严重降低特定角色的角色扮演性能；它无法区分冗余知识和基本角色特征。我们提出Persona-Pruner，一个通过从单个描述中隔离特定角色的子网络来雕琢轻量级角色扮演模型的框架。我们的实验一致表明，Persona-Pruner在保留角色扮演性能方面比现有最先进的LLM剪枝技术有效得多，在RoleBench上使用LLM-as-a-judge评分，将性能下降从密集模型减少至多93.8%（相比最强基线），同时仍保持通用LLM能力。代码可在以下网址获取：此https URL。
英文摘要：
Language Models (LMs) have shown remarkable potential as role-playing chatbots, delivering consistent, stylized interactions when given a specification of a character or user persona. However, applying these capabilities to real-world applications (e.g., ecosystems with numerous NPCs interacting simultaneously) exposes a critical inefficiency due to the excessive computational cost. In this paper, we question the necessity of dedicating a full, generalist model to a single persona, hypothesizing that a specific character identity relies on only a fraction of the model's total capacity. We observe that naively pruning LMs often severely degrades the role-playing performance for a specific persona; it does not distinguish between redundant knowledge and essential character traits. We propose Persona-Pruner, a framework that sculpts a lightweight role-playing model by isolating persona-specific sub-networks from a single description. Our experiments consistently show that Persona-Pruner preserves role-playing performance substantially more effectively than existing state-of-the-art LLM pruning techniques, reducing the performance drop from the dense model by up to 93.8% over the strongest baseline on RoleBench in LLM-as-a-judge score, while still maintaining general LLM capabilities. Code is available at https://github.com/jsu-kim/Persona-Pruner.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
