arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.22017cs.LG

面向增材制造的领域自适应大语言模型

Domain Adapted Large Language Models for Additive Manufacturing

Department of Mechanical Engineering, Carnegie Mellon University(机械工程系,卡内基梅隆大学)

Peter Pak, Amir Barati Farimani

AI总结:

本文通过约5000万token的小型数据集对开源大语言模型进行领域自适应预训练和指令微调,构建多模态领域自适应模型,在增材制造基准测试中达到90%以上准确率。

摘要:

本文提出了一系列多模态领域自适应大语言模型,这些模型基于指令微调变体的开源权重模型(Gemma 3、Qwen 3、Gemma 4),使用约5000万token的相对小型数据集构建。该数据集包含开放获取的增材制造期刊文章,从中提取数据用于领域自适应预训练和视觉指令微调过程。开发模型的各个阶段使用增材制造基准进行评估,该基准由增材制造领域特定任务和已发布资源组成。领域自适应和指令微调模型在语言和视觉任务中均表现出色,在通用增材制造知识方面达到90%以上的准确率。这种领域自适应预训练和指令微调策略为大语言模型在增材制造等领域的专业化提供了一种可访问的方法。

英文摘要:

This work presents a collection of multi-modal domain adapted large language models built upon the instruction tuned variants of open weight models (Gemma 3, Qwen 3, Gemma 4) using a relatively small dataset of around 50 million tokens. The dataset consists of open-access additive manufacturing journal articles with data extracted for the domain adaptive pretraining and visual instruction tuning processes. Various stages of the developed model are evaluated with the Additive-Manufacturing-Benchmark which consists of additive manufacturing domain specific tasks compiled published resources. Domain adapted and instruction tuned models exhibit proficiency in both language and vision based tasks, achieving accuracies upwards of 90% in general additive manufacturing knowledge. This domain adaptive pretraining and instruction tuning strategy outline an accessible specialization method for large language models to a domain such as additive manufacturing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑