Zhongjing：通过专家反馈和真实世界多轮对话增强大型语言模型的中医能力
Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialogue
AI总结：
针对中医领域LLM性能不足，提出Zhongjing模型，采用持续预训练、SFT和RLHF完整训练流程，并构建7万真实医患对话数据集CMtMedQA，显著提升多轮对话和主动询问能力，实验表明性能优于基线，部分能力媲美ChatGPT。
摘要：
近年来，大型语言模型（LLM）在理解和响应用户意图方面取得了显著突破。然而，在中医等专业领域，它们的性能落后于通用场景。现有的将中医融入LLM的工作依赖于使用单轮和蒸馏对话数据进行监督微调（SFT）。这些模型缺乏类似医生的主动询问和多轮理解能力，无法使响应与专家意图对齐。在这项工作中，我们介绍了Zhongjing，这是第一个基于LLaMA的中医大型语言模型，实现了从持续预训练、SFT到基于人类反馈的强化学习（RLHF）的完整训练流程。此外，我们构建了一个包含70,000个真实医患对话的中文多轮医疗对话数据集CMtMedQA，这显著增强了模型在复杂对话和主动询问发起方面的能力。我们还针对生物医学领域的独特特征定义了精细的标注规则和评估标准。大量实验结果表明，尽管参数规模相差100倍，Zhongjing在各种能力上优于基线，并在某些能力上与ChatGPT相当。消融研究也证明了每个组件的贡献：预训练增强了医学知识，RLHF进一步提高了指令遵循能力和安全性。我们的代码、数据集和模型可在https://github.com/SupritYoung/Zhongjing获取。
英文摘要：
Recent advances in Large Language Models (LLMs) have achieved remarkable breakthroughs in understanding and responding to user intents. However, their performance lag behind general use cases in some expertise domains, such as Chinese medicine. Existing efforts to incorporate Chinese medicine into LLMs rely on Supervised Fine-Tuning (SFT) with single-turn and distilled dialogue data. These models lack the ability for doctor-like proactive inquiry and multi-turn comprehension and cannot align responses with experts' intentions. In this work, we introduce Zhongjing, the first Chinese medical LLaMA-based LLM that implements an entire training pipeline from continuous pre-training, SFT, to Reinforcement Learning from Human Feedback (RLHF). Additionally, we construct a Chinese multi-turn medical dialogue dataset of 70,000 authentic doctor-patient dialogues, CMtMedQA, which significantly enhances the model's capability for complex dialogue and proactive inquiry initiation. We also define a refined annotation rule and evaluation criteria given the unique characteristics of the biomedical domain. Extensive experimental results show that Zhongjing outperforms baselines in various capacities and matches the performance of ChatGPT in some abilities, despite the 100x parameters. Ablation studies also demonstrate the contributions of each component: pre-training enhances medical knowledge, and RLHF further improves instruction-following ability and safety. Our code, datasets, and models are available at https://github.com/SupritYoung/Zhongjing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
