生成式AI时代面向“GPU匮乏者”的自动化文本评分
Automated Text Scoring in the Age of Generative AI for the GPU-poor
AI总结：
本文面向“GPU匮乏者”评估开源小规模GLM在自动化文本评分中的性能与效率，证明微调可获得可用但非SOTA的效果，并初步探索其生成评分反馈的能力。
摘要：
当前关于将生成式语言模型（GLM）用于自动化文本评分（ATS）的研究，几乎完全集中于通过应用程序编程接口（API）查询专有模型。然而，此类做法引发了透明度和安全性方面的问题，而且这些方法在效率或可定制性方面几乎没有优势。随着近期较小规模开源模型的激增，人们可以选择使用配备普通消费级硬件的计算机来探索GLM，也就是为“GPU匮乏者”服务。在本研究中，我们分析了用于ATS的开源小规模GLM的性能与效率。结果表明，GLM经过微调后可以达到足够好的性能，尽管并非最先进水平。除ATS之外，我们还通过提示GLM解释其评分，在分析模型生成反馈的能力方面迈出了一小步。模型生成的反馈展现出前景，但仍需围绕目标用例进行更严格的评估。
英文摘要：
Current research on generative language models (GLMs) for automated text scoring (ATS) has focused almost exclusively on querying proprietary models via Application Programming Interfaces (APIs). Yet such practices raise issues around transparency and security, and these methods offer little in the way of efficiency or customizability. With the recent proliferation of smaller, open-source models, there is the option to explore GLMs with computers equipped with modest, consumer-grade hardware, that is, for the "GPU poor." In this study, we analyze the performance and efficiency of open-source, small-scale GLMs for ATS. Results show that GLMs can be fine-tuned to achieve adequate, though not state-of-the-art, performance. In addition to ATS, we take small steps towards analyzing models' capacity for generating feedback by prompting GLMs to explain their scores. Model-generated feedback shows promise, but requires more rigorous evaluation focused on targeted use cases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
