FOCUS：解耦大语言模型中的专家角色以提升领域专家能力

中文摘要 AI 辅助

大语言模型（LLMs）可展现出多样化的角色，激活专家角色已被证实能提升领域专业度与任务准确率。然而，现有的角色控制方法常存在跨领域耦合问题，可能导致在医疗等高谨慎领域出现过度激进的行为，或在金融交易等风险敏感领域出现过度保守的情况。为解决该问题，我们提出FOCUS（用于解耦角色的正交控制微调，Fine-tuning with Orthogonal Control for Uncoupled Personas）。FOCUS首先自动从LLMs中提取专家角色向量，接着应用正交分解以解耦特定领域的专家角色，最后引入专家门控模块，根据任务上下文自适应控制角色激活。通过两阶段训练策略与门控选择正则化，模型学会为单领域及跨领域任务激活合适的角色。在金融、法律、医疗及跨领域基准上的实验表明，FOCUS提升了任务准确率，且优于现有的角色控制方法。我们的代码可在指定网址获取。

英文摘要

Large Language Models (LLMs) can exhibit diverse personas, and activating expert personas has been shown to improve domain expertise and task accuracy. However, existing persona control methods often suffer from cross-domain coupling, which may lead to overly aggressive behavior in high-caution domains such as healthcare, or excessive conservatism in risk-sensitive domains such as financial trading. To address this issue, we propose FOCUS (\textbf{\underline{F}}ine-tuning with \textbf{\underline{O}}rthogonal \textbf{\underline{C}}ontrol for \textbf{\underline{U}}ncoupled persona\textbf{\underline{S}}). FOCUS first automatically extracts expert persona vectors from LLMs, then applies orthogonal decomposition to decouple domain-specific expert personas, and finally introduces an expert gating module to adaptively control persona activation according to task contexts. With a two-stage training strategy and a gated selection regularizer, the model learns to activate appropriate personas for both single-domain and cross-domain tasks. Experiments on financial, legal, medical, and cross-domain benchmarks show that FOCUS improves task accuracy and outperforms existing persona control methods. Our code is available at \href{https://anonymous.4open.science/r/openpersona-48F4}{this url}.

发表机构

• Peking University（北京大学）
• The University of Sydney（悉尼大学）

机构由 AI 辅助整理，请以论文原文为准。