微调思维：利用思维链推理实现工业资产健康监测
Fine-Tuned Thoughts: Leveraging Chain-of-Thought Reasoning for Industrial Asset Health Monitoring
AI总结：
针对工业4.0领域小型语言模型复杂推理能力不足的问题，提出基于思维链蒸馏的工业资产健康知识蒸馏框架，通过MCQA提示蒸馏LLM能力并结合上下文学习验证，微调后的SLM性能大幅提升，缩小了与LLM的差距，相关代码已开源。
摘要：
小型语言模型（SLMs）凭借高效性、更低的计算需求以及可针对特定领域任务微调的特性，在工业应用等专业领域日益普及，能够提供精准且高性价比的解决方案。然而，在工业4.0这类专业领域中，利用SLMs执行复杂推理仍颇具挑战。本文提出了一种面向工业资产健康的知识蒸馏框架，通过思维链（CoT）蒸馏将大型语言模型（LLMs）的推理能力迁移到体积更小、效率更高的SLMs中。我们探讨了利用多项选择题问答（MCQA）提示蒸馏LLMs的优势与流程，以此增强推理能力并优化决策制定。我们还开展了上下文学习以验证生成知识的质量，并将搭载生成知识的微调SLMs的性能与广泛使用的LLMs进行基准对比。结果表明，具备CoT推理能力的微调SLMs性能远超基础模型，缩小了其与对应LLMs之间的差距。我们的代码已开源至：https://github.com/IBM/FailureSensorIQ。
英文摘要：
Small Language Models (SLMs) are becoming increasingly popular in specialized fields, such as industrial applications, due to their efficiency, lower computational requirements, and ability to be fine-tuned for domain-specific tasks, enabling accurate and cost-effective solutions. However, performing complex reasoning using SLMs in specialized fields such as Industry 4.0 remains challenging. In this paper, we propose a knowledge distillation framework for industrial asset health, which transfers reasoning capabilities via Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) to smaller, more efficient models (SLMs). We discuss the advantages and the process of distilling LLMs using multi-choice question answering (MCQA) prompts to enhance reasoning and refine decision-making. We also perform in-context learning to verify the quality of the generated knowledge and benchmark the performance of fine-tuned SLMs with generated knowledge against widely used LLMs. The results show that the fine-tuned SLMs with CoT reasoning outperform the base models by a significant margin, narrowing the gap to their LLM counterparts. Our code is open-sourced at: https://github.com/IBM/FailureSensorIQ.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
