面向领域自适应的大语言模型微调：训练策略、规模扩展、模型合并与协同能力的探索
Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities
AI总结：
本文系统探索了CPT、SFT、DPO和ORPO等微调策略对大语言模型领域自适应的影响，发现模型合并能产生超越单一模型的涌现能力，并表明模型规模是能力涌现的关键因素。
摘要：
大语言模型（LLMs）在材料科学与工程等领域应用的发展，依赖于能够使模型具备专业化技术能力的微调策略。在本工作中，我们探索了持续预训练（CPT）、监督微调（SFT）以及多种基于偏好的优化方法（包括直接偏好优化（DPO）和优势比偏好优化（ORPO））对微调后LLM性能的影响。我们的分析展示了这些策略如何影响模型结果，并揭示出多个微调模型的合并可以导致超越父模型各自贡献的能力涌现。我们发现模型合并带来了任一父模型都无法单独实现的新功能，从而在领域特定评估中取得更好的性能。我们给出了使用不同模型架构的实验，包括Llama 3.1 8B和Mistral 7B模型，并观察到类似行为。为探究这些结果是否同样适用于更小的模型，我们使用了一个具有17亿参数的微型LLM，并表明非常小的LLM在模型合并下不一定表现出涌现能力，这表明模型规模扩展可能是一个关键因素。在人类与AI模型之间开放式但一致的聊天对话中，我们的评估揭示了对不同模型变体表现的详细洞察，并表明最小模型在推理深度、创造力、清晰度和定量精度等关键指标上取得了较高的智能得分。其他实验包括基于不同的生物材料设计概念开发生成图像提示，以创建新的微观结构、建筑概念以及基于生物材料启发的建造原则的城市设计。
英文摘要：
The advancement of Large Language Models (LLMs) for domain applications in fields such as materials science and engineering depends on the development of fine-tuning strategies that adapt models for specialized, technical capabilities. In this work, we explore the effects of Continued Pretraining (CPT), Supervised Fine-Tuning (SFT), and various preference-based optimization approaches, including Direct Preference Optimization (DPO) and Odds Ratio Preference Optimization (ORPO), on fine-tuned LLM performance. Our analysis shows how these strategies influence model outcomes and reveals that the merging of multiple fine-tuned models can lead to the emergence of capabilities that surpass the individual contributions of the parent models. We find that model merging leads to new functionalities that neither parent model could achieve alone, leading to improved performance in domain-specific assessments. Experiments with different model architectures are presented, including Llama 3.1 8B and Mistral 7B models, where similar behaviors are observed. Exploring whether the results hold also for much smaller models, we use a tiny LLM with 1.7 billion parameters and show that very small LLMs do not necessarily feature emergent capabilities under model merging, suggesting that model scaling may be a key component. In open-ended yet consistent chat conversations between a human and AI models, our assessment reveals detailed insights into how different model variants perform and show that the smallest model achieves a high intelligence score across key criteria including reasoning depth, creativity, clarity, and quantitative precision. Other experiments include the development of image generation prompts based on disparate biological material design concepts, to create new microstructures, architectural concepts, and urban design based on biological materials-inspired construction principles.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
