考察不同领域知识库包含级别对基于GPT的智能导师的影响
Examining the Influence of Varied Levels of Domain Knowledge Base Inclusion in GPT-based Intelligent Tutors
AI总结：
本研究探讨了在GPT-4智能导师中集成不同级别知识库对提升回答准确性和教学能力的影响，发现知识库访问能提高准确性，且在教师语言和学生理解方面优于专家，但帮助学生的能力仍显不足。
摘要：
大型语言模型（LLMs）的最新进展促进了具有复杂对话能力的聊天机器人的发展。然而，LLMs对查询经常给出不准确的回答，这阻碍了其在教育环境中的应用。在本文中，我们研究了将知识库（KB）与LLM智能导师集成以提高响应可靠性的有效性。为此，我们设计了一个可扩展的KB，使教育主管能够无缝集成课程大纲，并由智能辅导系统自动处理。然后，我们详细介绍了一项评估，其中学生参与者被提问关于人工智能课程的问题。具有不同KB访问层级的GPT-4智能导师和人类领域专家对这些回答进行了评估。最后，学生们对智能导师的回应与领域专家的回应进行了交叉审查，并对他们的各种教学能力进行了排名。结果表明，尽管这些智能导师的准确性仍低于领域专家，但当授予KB访问权限时，智能导师的准确性有所提高。我们还观察到，具有KB访问权限的智能导师在像老师一样说话和理解学生方面表现出比领域专家更好的教学能力，而他们帮助学生方面的能力仍落后于领域专家。
英文摘要：
Recent advancements in large language models (LLMs) have facilitated the development of chatbots with sophisticated conversational capabilities. However, LLMs exhibit frequent inaccurate responses to queries, hindering applications in educational settings. In this paper, we investigate the effectiveness of integrating a knowledge base (KB) with LLM intelligent tutors to increase response reliability. To achieve this, we design a scaleable KB that affords educational supervisors seamless integration of lesson curricula, which is automatically processed by the intelligent tutoring system. We then detail an evaluation, where student participants were presented with questions about the artificial intelligence curriculum to respond to. GPT-4 intelligent tutors with varying hierarchies of KB access and human domain experts then assessed these responses. Lastly, students cross-examined the intelligent tutors' responses to the domain experts' and ranked their various pedagogical abilities. Results suggest that, although these intelligent tutors still demonstrate a lower accuracy compared to domain experts, the accuracy of the intelligent tutors increases when access to a KB is granted. We also observe that the intelligent tutors with KB access exhibit better pedagogical abilities to speak like a teacher and understand students than those of domain experts, while their ability to help students remains lagging behind domain experts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
