EduMod-LLM: 一种用于设计灵活且透明教育助手的模块化方法
EduMod-LLM: A Modular Approach for Designing Flexible and Transparent Educational Assistants
AI总结：
EduMod-LLM通过模块化方法评估教育问答系统各组件性能，揭示失败模式，提升透明度和教学对齐效果。
摘要：
随着基于大语言模型（LLM）的问答（QA）系统在教育领域的广泛应用，评估其在各个管道组件上的性能变得至关重要。在本工作中，我们引入了{\model}，一种模块化功能调用LLM管道，并沿三个关键轴进行综合评估：功能调用策略、检索方法和生成语言模型。我们的框架通过隔离和评估每个组件实现细粒度分析。我们对LLM进行功能调用性能基准测试，比较我们的新型结构感知检索方法与向量基和LLM评分基线，并评估各种LLM进行响应合成。这种模块化方法揭示了特定的失败模式和性能模式，支持开发可解释且有效的教育QA系统。我们的发现证明了模块化功能调用在提高系统透明度和教学对齐方面的价值。网站和补充材料：https://chancharikmitra.github.io/EduMod-LLM-website/
英文摘要：
With the growing use of Large Language Model (LLM)-based Question-Answering (QA) systems in education, it is critical to evaluate their performance across individual pipeline components. In this work, we introduce {\model}, a modular function-calling LLM pipeline, and present a comprehensive evaluation along three key axes: function calling strategies, retrieval methods, and generative language models. Our framework enables fine-grained analysis by isolating and assessing each component. We benchmark function-calling performance across LLMs, compare our novel structure-aware retrieval method to vector-based and LLM-scoring baselines, and evaluate various LLMs for response synthesis. This modular approach reveals specific failure modes and performance patterns, supporting the development of interpretable and effective educational QA systems. Our findings demonstrate the value of modular function calling in improving system transparency and pedagogical alignment. Website and Supplementary Material: https://chancharikmitra.github.io/EduMod-LLM-website/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
