基于语言模型的K-12教育试题生成提示策略：弥合小型与大型语言模型之间的差距
Prompting Strategies for Language Model-Based Item Generation in K-12 Education: Bridging the Gap Between Small and Large Language Models
AI总结：
本研究比较微调中型模型Gemma与大型模型GPT-3.5在K-12形态学多选题自动生成中的表现，发现结合思维链与顺序设计的结构化提示策略能显著提升中型模型生成质量，为有限数据条件下的试题自动生成提供了实用可扩展的工作流程。
摘要：
本研究探索使用语言模型进行自动生成（AIG），以创建用于形态学评估的多项选择题（MCQs），旨在降低人工测试开发的成本和不一致性。研究采用双重方法。首先，我们将一个微调的中等模型（Gemma，2B）与一个更大的未微调模型（GPT-3.5，175B）进行比较。其次，我们评估了七种结构化提示策略，包括零样本、少样本、思维链、基于角色、顺序以及组合策略。生成的试题使用自动化指标和专家评分在五个维度上进行评估。我们还使用在专家评分样本上训练的GPT-4.1来大规模模拟人工评分。结果表明，结构化提示，尤其是结合思维链和顺序设计的策略，显著改善了Gemma的输出。Gemma通常比GPT-3.5的零样本响应产生更符合构念且教学上更合适的试题，提示设计在中等规模模型性能中发挥关键作用。本研究表明，结构化提示和高效微调可以在有限数据条件下增强中等规模模型的AIG能力。我们强调结合自动化指标、专家判断和大模型模拟以确保与评估目标一致的价值。所提出的工作流程为K-12语言评估试题的开发和验证提供了一种实用且可扩展的方法。
英文摘要：
This study explores automatic generation (AIG) using language models to create multiple choice questions (MCQs) for morphological assessment, aiming to reduce the cost and inconsistency of manual test development. The study used a two-fold approach. First, we compared a fine-tuned medium model (Gemma, 2B) with a larger untuned one (GPT-3.5, 175B). Second, we evaluated seven structured prompting strategies, including zero-shot, few-shot, chain-of-thought, role-based, sequential, and combinations. Generated items were assessed using automated metrics and expert scoring across five dimensions. We also used GPT-4.1, trained on expert-rated samples, to simulate human scoring at scale. Results show that structured prompting, especially strategies combining chain-of-thought and sequential design, significantly improved Gemma's outputs. Gemma generally produced more construct-aligned and instructionally appropriate items than GPT-3.5's zero-shot responses, with prompt design playing a key role in mid-size model performance. This study demonstrates that structured prompting and efficient fine-tuning can enhance midsized models for AIG under limited data conditions. We highlight the value of combining automated metrics, expert judgment, and large-model simulation to ensure alignment with assessment goals. The proposed workflow offers a practical and scalable way to develop and validate language assessment items for K-12.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
