微调ChatGPT用于中文书面科学解释的自动评分
Fine-tuning ChatGPT for Automatic Scoring of Written Scientific Explanations in Chinese
AI总结：
本研究微调ChatGPT用于中文科学解释自动评分，发现其经领域适配后可有效评分，且评分准确率与推理复杂度、语言特征相关，为中文教育评估的智能评分模型优化提供依据。
摘要：
对科学现象的解释能力培养是科学评估的核心环节，但为学生撰写的解释评分既具挑战性又耗费大量资源。大语言模型（LLM）已在解决该问题上展现出潜力，尤其在英语等字母语言场景中。不过，它们在表意语言中的适用性仍有待探索。本研究探究了微调主流大语言模型ChatGPT用于自动评分中文科学解释的潜力。研究收集了学生对7项科学解释任务的作答并进行自动评分，采用Kendall相关系数分析评分准确率与推理复杂度的关联，并通过定性分析探究语言特征对评分准确率的影响机制。结果显示，领域特定适配使ChatGPT能够准确评分中文科学解释，但评分准确率与推理复杂度存在相关性：低水平作答呈负相关，高水平作答呈正相关。该模型会高估句式复杂的低水平作答中的复杂推理，低估使用简洁因果推理的高水平作答。这些相关性源于语言特征：简洁清晰可提升低水平作答的评分准确率，而全面性可提升高水平作答的评分准确率。更简短的低水平作答往往评分更准确，而信息丰富的较长高水平作答评分准确率更高。这些发现证明了大语言模型在中文语境下自动评分的有效性，同时强调了语言特征与推理复杂度在微调教育评估评分模型中的重要性。
英文摘要：
The development of explanations for scientific phenomena is essential in science assessment, but scoring student-written explanations remains challenging and resource-intensive. Large language models (LLMs) have shown promise in addressing this issue, particularly in alphabetic languages like English. However, their applicability to logographic languages is less explored. This study investigates the potential of fine-tuning ChatGPT, a leading LLM, to automatically score scientific explanations written in Chinese. Student responses to seven scientific explanation tasks were collected and automatically scored, with scoring accuracy examined in relation to reasoning complexity using the Kendall correlation. A qualitative analysis explored how linguistic features influenced scoring accuracy. The results show that domain-specific adaptation enables ChatGPT to score Chinese scientific explanations with accuracy. However, scoring accuracy correlates with reasoning complexity: a negative correlation for lower-level responses and a positive one for higher-level responses. The model overrates complex reasoning in low-level responses with intricate sentence structures and underrates high-level responses using concise causal reasoning. These correlations stem from linguistic features--simplicity and clarity enhance accuracy for lower-level responses, while comprehensiveness improves accuracy for higher-level ones. Simpler, shorter responses tend to score more accurately at lower levels, whereas longer, information-rich responses yield better accuracy at higher levels. These findings demonstrate the effectiveness of LLMs in automatic scoring within a Chinese context and emphasize the importance of linguistic features and reasoning complexity in fine-tuning scoring models for educational assessments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
