CollectiveSFT：利用医疗保健中的集体指令扩展大型语言模型以应对中文医学基准
CollectiveSFT: Scaling Large Language Models for Chinese Medical Benchmark with Collective Instructions in Healthcare
AI总结：
本研究提出CollectiveSFT方法，通过构建多样且分布良好的中文医学指令数据集进行监督微调，使较小模型在CMB基准上达到与较大模型相当的性能，证明数据集质量与多样性比模型规模更关键。
摘要：
大型语言模型（LLMs）的快速进展促使人们创建了大量基准来评估其能力。本研究聚焦于中文综合医学基准（CMB），展示了监督微调（SFT）中数据集的多样性和分布如何提升LLM性能。值得注意的是，我们成功训练了一个较小的基础模型，使其取得了与较大模型相当的分数，这表明多样且分布良好的数据集可以优化性能，而与模型规模无关。本研究表明，即使是较小的模型，只要使用精心策划且多样化的数据集，也可能达到较高的性能水平。通过整合广泛的指令内容，我们的方法解决了数据质量不一致等潜在问题。我们的结果意味着，更广泛的训练数据范围可能增强模型在不同医学场景中的泛化能力和有效执行能力，凸显了数据集质量和多样性在微调过程中的重要性。我们在 https://github.com/CAS-SIAT-XinHai/CollectiveSFT 开源该模型以供未来研究使用。
英文摘要：
The rapid progress in Large Language Models (LLMs) has prompted the creation of numerous benchmarks to evaluate their capabilities.This study focuses on the Comprehensive Medical Benchmark in Chinese (CMB), showcasing how dataset diversity and distribution in supervised fine-tuning (SFT) may enhance LLM performance.Remarkably, We successfully trained a smaller base model to achieve scores comparable to larger models, indicating that a diverse and well-distributed dataset can optimize performance regardless of model size.This study suggests that even smaller models may reach high performance levels with carefully curated and varied datasets. By integrating a wide range of instructional content, our approach addresses potential issues such as data quality inconsistencies. Our results imply that a broader spectrum of training data may enhance a model's ability to generalize and perform effectively across different medical scenarios, highlighting the importance of dataset quality and diversity in fine-tuning processes. We open-source the model for future research at https://github.com/CAS-SIAT-XinHai/CollectiveSFT
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
