大型语言模型能否区分有害与论辩性文章？迈向道德文章评分的步骤
Can Large Language Models Differentiate Harmful from Argumentative Essays? Steps Toward Ethical Essay Scoring
AI总结：
本研究提出HED基准，揭示LLMs在识别有害文章方面的能力不足，并强调需开发更注重伦理因素的AES系统。
摘要：
本研究针对自动文章评分（AES）系统和大型语言模型（LLMs）在有效识别和评分有害文章方面存在的关键缺陷进行了探讨。尽管在AES技术方面取得了进展，但当前模型往往忽视文章中存在伦理和道德问题的元素，错误地对可能传播有害观点的文章给予高分。在本研究中，我们引入了有害文章检测（HED）基准，其中包括整合了敏感主题如种族主义和性别偏见的文章，以测试各种LLMs在识别和评分有害内容方面的有效性。我们的发现表明：（1）LLMs需要进一步改进以准确区分有害和论辩性文章；（2）当前的AES模型和LLMs在评分时均未考虑内容的伦理维度。本研究强调了开发更加稳健的AES系统的重要性，这些系统应能敏感地考虑到其评分内容的伦理影响。
英文摘要：
This study addresses critical gaps in Automated Essay Scoring (AES) systems and Large Language Models (LLMs) with regard to their ability to effectively identify and score harmful essays. Despite advancements in AES technology, current models often overlook ethically and morally problematic elements within essays, erroneously assigning high scores to essays that may propagate harmful opinions. In this study, we introduce the Harmful Essay Detection (HED) benchmark, which includes essays integrating sensitive topics such as racism and gender bias, to test the efficacy of various LLMs in recognizing and scoring harmful content. Our findings reveal that: (1) LLMs require further enhancement to accurately distinguish between harmful and argumentative essays, and (2) both current AES models and LLMs fail to consider the ethical dimensions of content during scoring. The study underscores the need for developing more robust AES systems that are sensitive to the ethical implications of the content they are scoring.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
