LLM驱动的个性化答案生成与评估
LLM-Driven Personalized Answer Generation and Evaluation
AI总结：
本研究探索LLM生成个性化学习答案的潜力，搭建验证框架与数据集，在语言学习、编程领域测试不同提示策略，经多维度评估证实提供示例可显著提升LLM的个性化回答能力。
摘要：
在线学习凭借其灵活性与可及性实现了快速发展。适配个体学习者需求的个性化服务对提升学习体验至关重要，在在线学习场景中尤为如此。个性化的一个核心方面是为学习者提供针对其特定问题定制的答案。因此，本文探究了大语言模型（Large Language Models, LLMs）为学习者的问题生成个性化答案的潜力，以此提升学习参与度并减轻教育工作者的工作负担。为评估LLM在此场景下的有效性，我们借助StackExchange平台在语言学习和编程两个不同领域开展了综合研究。我们开发了一套框架和一个数据集，用于验证自动生成的个性化答案。随后，我们采用不同策略生成个性化答案，涵盖0-shot、1-shot和few-shot场景。我们通过三种方法评估生成的答案：1. BERTScore，2. LLM评估，3. 人类评估。研究结果表明，为LLM提供期望答案的示例（来自该学习者或相似学习者），能够显著提升LLM根据个体学习者需求定制回复的能力。
英文摘要：
Online learning has experienced rapid growth due to its flexibility and accessibility. Personalization, adapted to the needs of individual learners, is crucial for enhancing the learning experience, particularly in online settings. A key aspect of personalization is providing learners with answers customized to their specific questions. This paper therefore explores the potential of Large Language Models (LLMs) to generate personalized answers to learners' questions, thereby enhancing engagement and reducing the workload on educators. To evaluate the effectiveness of LLMs in this context, we conducted a comprehensive study using the StackExchange platform in two distinct areas: language learning and programming. We developed a framework and a dataset for validating automatically generated personalized answers. Subsequently, we generated personalized answers using different strategies, including 0-shot, 1-shot, and few-shot scenarios. The generated answers were evaluated using three methods: 1. BERTScore, 2. LLM evaluation, and 3. human evaluation. Our findings indicated that providing LLMs with examples of desired answers (from the learner or similar learners) can significantly enhance the LLMs' ability to tailor responses to individual learners' needs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
