APT：通过弱点案例获取与迭代偏好训练提升专家大语言模型性能
APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training
AI总结：
本文提出 APT 方法，通过自生成错误案例和相似案例进行迭代偏好训练，在提升大语言模型领域性能的同时保持通用能力不下降。
摘要：
大语言模型（LLMs）通常需要针对特定任务进行领域微调，但这存在削弱其通用能力的风险。在领域特定增强与通用模型效用之间保持平衡是一个关键挑战。本文提出了一种名为 APT（弱点案例获取与迭代偏好训练）的新方法，利用自生成的非偏好弱点数据（错误案例和相似案例）来增强领域特定性能。APT 的独特之处在于仅使用发生错误的样本以及为此检索到的一小组相似样本进行训练。这种有针对性的训练最大限度地减少了对模型现有知识库的干扰，从而有效保留通用能力。在 LLama-2 和 Mistral-V0.3 模型上跨多个基准的实验结果表明，APT 确保通用能力不降低，并且与多种现有方法相比，在下游任务上取得了更优的性能。这验证了我们的方法是一种在不牺牲模型更广泛适用性的前提下增强领域特定能力的有效策略。
英文摘要：
Large Language Models (LLMs) often require domain-specific fine-tuning to address targeted tasks, which risks degrading their general capabilities. Maintaining a balance between domain-specific enhancements and general model utility is a key challenge. This paper proposes a novel approach named APT (Weakness Case Acquisition and Iterative Preference Training) to enhance domain-specific performance with self-generated dis-preferred weakness data (bad cases and similar cases). APT uniquely focuses on training the model using only those samples where errors occur, alongside a small, similar set of samples retrieved for this purpose. This targeted training minimizes interference with the model's existing knowledge base, effectively retaining generic capabilities. Experimental results on the LLama-2 and Mistral-V0.3 models across various benchmarks demonstrate that APT ensures no reduction in generic capacity and achieves superior performance on downstream tasks compared to various existing methods. This validates our method as an effective strategy for enhancing domain-specific capabilities without sacrificing the model's broader applicability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
