为燃烧科学中的大语言模型知识注入和评估构建统一的基础框架
A unified foundational framework for knowledge injection and evaluation of Large Language Models in Combustion Science
- School of Mechanics and Engineering Science, Peking University（力学与工程科学学院，北京大学）
- AI for Science Institute（人工智能科学研究院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究提出一个统一框架，用于开发燃烧科学专用的大语言模型，通过三阶段知识注入路径提升模型性能。
AI中文摘要：
为了推动基础大语言模型（LLMs）在燃烧科学中的发展，本研究提出了首个端到端的框架，用于开发针对燃烧领域的专用模型。该框架包括一个规模达35亿词的AI-ready多模态知识库，提取自超过20万篇同行评审文章、8000篇论文和大约40万行燃烧CFD代码；一个严格且大部分自动化评估基准（CombustionQA，涵盖八个子领域共436个问题）；以及一个三阶段的知识注入路径，从轻量级检索增强生成（RAG）逐步发展到知识图谱增强检索和持续预训练。我们首先定量验证了第一阶段（朴素RAG）并发现存在硬性限制：标准RAG的准确率峰值为60%，远超零样本性能（23%）但远低于理论上限（87%）。我们进一步证明，该阶段的性能严重受限于上下文污染。因此，构建领域基础模型需要结构化的知识图谱和持续预训练（第二和第三阶段）。
英文摘要：
To advance foundation Large Language Models (LLMs) for combustion science, this study presents the first end-to-end framework for developing domain-specialized models for the combustion community. The framework comprises an AI-ready multimodal knowledge base at the 3.5 billion-token scale, extracted from over 200,000 peer-reviewed articles, 8,000 theses and dissertations, and approximately 400,000 lines of combustion CFD code; a rigorous and largely automated evaluation benchmark (CombustionQA, 436 questions across eight subfields); and a three-stage knowledge-injection pathway that progresses from lightweight retrieval-augmented generation (RAG) to knowledge-graph-enhanced retrieval and continued pretraining. We first quantitatively validate Stage 1 (naive RAG) and find a hard ceiling: standard RAG accuracy peaks at 60%, far surpassing zero-shot performance (23%) yet well below the theoretical upper bound (87%). We further demonstrate that this stage's performance is severely constrained by context contamination. Consequently, building a domain foundation model requires structured knowledge graphs and continued pretraining (Stages 2 and 3).
