arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.03008cs.AI
收藏

知识图谱作为基于LLM的学习推荐解释的上下文来源

Knowledge Graphs as Context Sources for LLM-Based Explanations of Learning Recommendations

University of Siegen(锡根大学)

Hasan Abu-Rasheed, Christian Weber, Madjid Fathi

AI总结:

本文提出利用知识图谱作为LLM提示的上下文来源,为学习推荐生成精确解释,降低幻觉风险,实验显示相比纯GPT模型在召回率和精确率上均有提升。

摘要:

在个性化教育时代,为学习推荐提供可理解的解释对于增强学习者对推荐学习内容的理解和参与度具有重要价值。大型语言模型(LLMs)和广义上的生成式人工智能最近为生成类人解释开辟了新途径,用于伴随学习推荐或为其生成解释。然而,在教育这一敏感领域,其精确性仍远未达到可接受的水平。为利用LLMs的能力,同时确保对学习者意图的高度精确性,本文提出了一种方法,将知识图谱(KG)作为LLM提示的事实上下文来源,以降低模型幻觉的风险,防止错误或不精确信息的产生,同时保持应用预期的学习上下文。我们利用知识图谱中的语义关系来提供关于学习推荐的精选知识。在领域专家参与的情况下,我们将解释设计为文本模板,由LLM填充和完善。作为研究的一部分,领域专家被整合到提示工程阶段,以确保解释包含与学习者相关的信息。我们使用Rouge-N和Rouge-L指标定量评估我们的方法,并与专家和学习者进行定性评估。结果表明,与仅由GPT模型生成的解释相比,我们生成的解释在召回率和精确率上均有提升,同时在最终学习解释中生成不精确信息的风险大大降低。

英文摘要:

In the era of personalized education, the provision of comprehensible explanations for learning recommendations is of a great value to enhance the learner's understanding and engagement with the recommended learning content. Large language models (LLMs) and generative AI in general have recently opened new doors for generating human-like explanations, for and along learning recommendations. However, their precision is still far away from acceptable in a sensitive field like education. To harness the abilities of LLMs, while still ensuring a high level of precision towards the intent of the learners, this paper proposes an approach to utilize knowledge graphs (KG) as a source of factual context, for LLM prompts, reducing the risk of model hallucinations, and safeguarding against wrong or imprecise information, while maintaining an application-intended learning context. We utilize the semantic relations in the knowledge graph to offer curated knowledge about learning recommendations. With domain-experts in the loop, we design the explanation as a textual template, which is filled and completed by the LLM. Domain experts were integrated in the prompt engineering phase as part of a study, to ensure that explanations include information that is relevant to the learner. We evaluate our approach quantitatively using Rouge-N and Rouge-L measures, as well as qualitatively with experts and learners. Our results show an enhanced recall and precision of the generated explanations compared to those generated solely by the GPT model, with a greatly reduced risk of generating imprecise information in the final learning explanation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑