IB-GRPO: 通过基于指标的群体相对策略优化对基于大语言模型的学习路径推荐进行对齐
IB-GRPO: Aligning LLM-based Learning Path Recommendation with Educational Objectives via Indicator-Based Group Relative Policy Optimization
AI总结：
IB-GRPO通过基于指标的群体相对策略优化，解决大语言模型在学习路径推荐中的多目标对齐问题，提升学习效果与教学目标的一致性。
摘要：
学习路径推荐（LPR）旨在生成个性化学习项序列，以最大化长期学习效果，同时尊重教学原则和操作约束。尽管大语言模型（LLMs）在自由形式推荐中提供丰富的语义理解，但将其应用于长周期LPR具有挑战性，原因包括（i）与教学目标如最近发展区（ZPD）在稀疏、延迟反馈下的不一致，（ii）专家演示稀缺且成本高，以及（iii）学习效果、难度安排、长度可控性和轨迹多样性之间的多目标交互。为了解决这些问题，我们提出了IB-GRPO（基于指标的群体相对策略优化），一种用于基于LLM的LPR的指标引导对齐方法。为缓解数据稀缺问题，我们通过遗传算法搜索和教师RL代理构建混合专家演示，并通过监督微调预热LLM。在此预热基础上，我们设计了会话内ZPD对齐分数用于难度安排。IB-GRPO随后使用I_{ε+}主导指标计算多个目标上的群体相对优势，避免手动标量化并改善帕累托权衡。在ASSIST09和Junyi上使用KES模拟器进行实验，使用Qwen2.5-7B骨干，结果在代表性RL和LLM基线中表现出一致的改进。
英文摘要：
Learning Path Recommendation (LPR) aims to generate personalized sequences of learning items that maximize long-term learning effect while respecting pedagogical principles and operational constraints. Although large language models (LLMs) offer rich semantic understanding for free-form recommendation, applying them to long-horizon LPR is challenging due to (i) misalignment with pedagogical objectives such as the Zone of Proximal Development (ZPD) under sparse, delayed feedback, (ii) scarce and costly expert demonstrations, and (iii) multi-objective interactions among learning effect, difficulty scheduling, length controllability, and trajectory diversity. To address these issues, we propose IB-GRPO (Indicator-Based Group Relative Policy Optimization), an indicator-guided alignment approach for LLM-based LPR. To mitigate data scarcity, we construct hybrid expert demonstrations via Genetic Algorithm search and teacher RL agents and warm-start the LLM with supervised fine-tuning. Building on this warm-start, we design a within-session ZPD alignment score for difficulty scheduling. IB-GRPO then uses the $I_{ε+}$ dominance indicator to compute group-relative advantages over multiple objectives, avoiding manual scalarization and improving Pareto trade-offs. Experiments on ASSIST09 and Junyi using the KES simulator with a Qwen2.5-7B backbone show consistent improvements over representative RL and LLM baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
