WisdomBot:利用人工智能知识微调大型语言模型
WisdomBot: Tuning Large Language Models with Artificial Intelligence Knowledge
AI总结:
本文提出WisdomBot,结合教育理论与检索增强技术微调中文大语言模型,提升教育领域回答的专业性与可靠性。
摘要:
大型语言模型(LLMs)已成为自然语言处理(NLP)中的强大工具,展现出人工智能生成智能(AGI)的光明前景。尽管在通用领域表现出色,但由于该领域面临的独特挑战,如需要更专业的知识、要求个性化学习体验以及必须对复杂概念进行简明解释,LLMs在教育领域仍表现欠佳。为解决这些问题,本文提出了一种名为WisdomBot的新型教育领域LLM,它将LLMs的强大能力与教育理论相结合,使其能够无缝融入教育情境。具体而言,我们在布鲁姆分类法的指导下,利用自我指导的知识概念和指令作为训练数据。为了进一步提高模型在事实性问题回答上的准确性和专业性,我们在推理过程中引入了两项关键增强,即本地知识库检索增强和搜索引擎检索增强。我们通过将该方法应用于多个中文LLM来验证其有效性,从而展示微调后的模型能够生成更可靠、更专业的回答。
英文摘要:
Large language models (LLMs) have emerged as powerful tools in natural language processing (NLP), showing a promising future of artificial generated intelligence (AGI). Despite their notable performance in the general domain, LLMs have remained suboptimal in the field of education, owing to the unique challenges presented by this domain, such as the need for more specialized knowledge, the requirement for personalized learning experiences, and the necessity for concise explanations of complex concepts. To address these issues, this paper presents a novel LLM for education named WisdomBot, which combines the power of LLMs with educational theories, enabling their seamless integration into educational contexts. To be specific, we harness self-instructed knowledge concepts and instructions under the guidance of Bloom's Taxonomy as training data. To further enhance the accuracy and professionalism of model's response on factual questions, we introduce two key enhancements during inference, i.e., local knowledge base retrieval augmentation and search engine retrieval augmentation during inference. We substantiate the effectiveness of our approach by applying it to several Chinese LLMs, thereby showcasing that the fine-tuned models can generate more reliable and professional responses.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。