分析用于课堂讨论评估的大型语言模型
Analyzing Large Language Models for Classroom Discussion Assessment
AI总结：
本文研究了任务表述、上下文长度和少样本示例对两个大型语言模型课堂讨论评估性能的影响，并推荐了一种兼顾性能、效率与一致性的评估方法。
摘要：
随着大型语言模型（LLMs）等自然语言处理新进展的出现，自动评估课堂讨论质量正变得越来越可行。在这项工作中，我们考察了2个LLM的评估性能与3个可能影响性能的因素之间的相互作用：任务表述、上下文长度和少样本示例。我们还探讨了这2个LLM的计算效率和预测一致性。我们的结果表明，上述3个因素确实会影响所测试LLM的性能，并且一致性与性能之间存在关联。我们推荐一种基于LLM的评估方法，该方法在预测性能、计算效率和一致性之间具有良好的平衡。
英文摘要：
Automatically assessing classroom discussion quality is becoming increasingly feasible with the help of new NLP advancements such as large language models (LLMs). In this work, we examine how the assessment performance of 2 LLMs interacts with 3 factors that may affect performance: task formulation, context length, and few-shot examples. We also explore the computational efficiency and predictive consistency of the 2 LLMs. Our results suggest that the 3 aforementioned factors do affect the performance of the tested LLMs and there is a relation between consistency and performance. We recommend a LLM-based assessment approach that has a good balance in terms of predictive performance, computational efficiency, and consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
