Automated Essay Scoring systems have traditionally focused on holistic scores, limiting their pedagogical usefulness, especially in the case of complex essay genres such as argumentative writing. In educational contexts, teachers and learners require interpretable, trait-level feedback that aligns with instructional goals and established rubrics. In this paper, we study trait-based Automatic Argumentative Essay Scoring using two complementary modeling paradigms designed for realistic educational deployment: (1) structured in-context learning with small open-source LLMs, and (2) a supervised, encoder-based BigBird model with a CORAL-style ordinal regression formulation, optimized for long-sequence understanding. We conduct a systematic evaluation on the ASAP++ dataset, which includes essay scores across five quality traits, offering strong coverage of core argumentation dimensions. LLMs are prompted with designed, rubric-aligned in-context examples, along with feedback and confidence requests, while we explicitly model ordinality in scores with the BigBird model via the rank-consistent CORAL framework. Our results show that explicitly modeling score ordinality substantially improves agreement with human raters across all traits, outperforming LLMs and nominal classification and regression-based baselines. This finding reinforces the importance of aligning model objectives with rubric semantics for educational assessment. At the same time, small open-source LLMs achieve a competitive performance without task-specific fine-tuning, particularly for reasoning-oriented traits, while enabling transparent, privacy-preserving, and locally deployable assessment scenarios. Our findings provide methodological, modeling, and practical insights for the design of AI-based educational systems that aim to deliver interpretable, rubric-aligned feedback for argumentative writing.
WrAFT: a Modularized Automated Writing Evaluation System for Argumentative Essays
Department of Informatics, King’s College London（伦敦国王学院信息学系）；Waseda University（早稻田大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；University of Sheffield（谢菲尔德大学）；Hosei University（法政大学）
Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoring
National Key Laboratory for Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）；School of Computer Science, Peking University（北京大学计算机学院）；School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）
Argument Rarity-based Originality Assessment for AI-Assisted Writing
Ritsumeikan Global Innovation Research Organization (R-GIRO), Ritsumeikan University（日光全球创新研究组织（R-GIRO）、日光大学）；Faculty of Business and Commerce, Kansai University（大阪大学商学部）；Arbege Corporation（Arbege公司）；College of Life Sciences, Ritsumeikan University（日光大学生命科学学院）；Office of General Education, Ritsumeikan University（日光大学通识教育办公室）
PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback
Department of Educational Psychology, East China Normal University（华东师范大学教育心理学系）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University（华东师范大学上海智能教育研究院）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）
EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Beijing Future Brain Education Technology Co., Ltd.（北京未来脑教育科技有限公司）；The Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）；Guangxi Zhuang Autonomous Region Big Data Research Institute（广西壮族自治区大数据研究院）
Towards Comprehensive Argument Analysis in Education: Dataset, Tasks, and Method
Lab of Artificial Intelligence for Education, East China Normal University（教育人工智能实验室，华东师范大学）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University（教育人工智能上海研究院，华东师范大学）；School of Computer Science and Technology, East China Normal University（计算机科学与技术学院，华东师范大学）；Department of Chinese Language and Literature, East China Normal University（中文语言文学系，华东师范大学）；College of Education, Zhejiang University（教育学院，浙江大学）
Riiid Labs（Riiid实验室）；Riiid AI Research（Riiid人工智能研究院）
This paper surveys and organizes research works in an under-studied area, which we call automated evaluation for student argumentative writing. Unlike traditional automated writing evaluation that focuses on holistic essay scoring, this field is more specific: it focuses on evaluating argumentative essays and offers specific feedback, including argumentation structures, argument strength trait score, etc. The focused and detailed evaluation is useful for helping students acquire important argumentation skill. In this paper we organize existing works around tasks, data and methods. We further e…
The development of large and super-large language models, such as GPT-3, T5, Switch Transformer, ERNIE, etc., has significantly improved the performance of text generation. One of the important research directions in this area is the generation of texts with arguments. The solution of this problem can be used in business meetings, political debates, dialogue systems, for preparation of student essays. One of the main domains for these applications is the economic sphere. The key problem of the argument text generation for the Russian language is the lack of annotated argumentation corpora. In …
Columbia University（哥伦比亚大学）；Educational Testing Service（教育考试服务中心）
Argument mining is often addressed by a pipeline method where segmentation of text into argumentative units is conducted first and proceeded by an argument component identification task. In this research, we apply a token-level classification to identify claim and premise tokens from a new corpus of argumentative essays written by middle school students. To this end, we compare a variety of state-of-the-art models such as discrete features and deep learning architectures (e.g., BiLSTM networks and BERT-based architectures) to identify the argument components. We demonstrate that a BERT-based m…
BIG5-TPoT: Predicting BIG Five Personality Traits, Facets, and Items Through Targeted Preselection of Texts
The University of Maryland Applied Research Laboratory for Intelligence and Security (ARLIS)（马里兰大学应用研究实验室（情报与安全实验室））；The University of Maryland (UMD)（马里兰大学）
Big5PersonalityEssays: Introducing a Novel Synthetic Generated Dataset Consisting of Short State-of-Consciousness Essays Annotated Based on the Five Factor Model of Personality
University POLITEHNICA of Bucharest（布加勒斯特理工大学）；The National Institute for Research and Development in Informatics – ICI Bucharest（国家信息学研发研究所——布加勒斯特ICI）
Learning Research and Development Center（学习研究与发展中心）；University of Pittsburgh（匹兹堡大学）
Automated writing evaluation systems can improve students' writing insofar as students attend to the feedback provided and revise their essay drafts in ways aligned with such feedback. Existing research on revision of argumentative writing in such systems, however, has focused on the types of revisions students make (e.g., surface vs. content) rather than the extent to which revisions actually respond to the feedback provided and improve the essay. We introduce an annotation scheme to capture the nature of sentence-level revisions of evidence use and reasoning (the `RER' scheme) and apply it t…
Vyatka State University（维亚特卡国立大学）；ITMO University（圣彼得堡国立信息技术机械与光学大学）
The development of large and super-large language models, such as GPT-3, T5, Switch Transformer, ERNIE, etc., has significantly improved the performance of text generation. One of the important research directions in this area is the generation of texts with arguments. The solution of this problem can be used in business meetings, political debates, dialogue systems, for preparation of student essays. One of the main domains for these applications is the economic sphere. The key problem of the argument text generation for the Russian language is the lack of annotated argumentation corpora. In …
ChatGPT-generated texts show authorship traits that identify them as non-human
Università degli Studi di Pavia（帕维亚大学）；University of Birmingham（伯明翰大学）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；Institució Catalana de Recerca i Estudis Avançats (ICREA)（加泰罗尼亚高等研究院）
Most research in the area of automatic essay grading (AEG) is geared towards scoring the essay holistically while there has also been some work done on scoring individual essay traits. In this paper, we describe a way to score essays holistically using a multi-task learning (MTL) approach, where scoring the essay holistically is the primary task, and scoring the essay traits is the auxiliary task. We compare our results with a single-task learning (STL) approach, using both LSTMs and BiLSTMs. We also compare our results of the auxiliary task with such tasks done in other AEG systems. To find o…
Federal University of Piauí（皮奥伊联邦大学）；Federal Institute of Piauí（皮奥伊联邦学院）
Automatic Essay Scoring (AES) is defined as the computer technology that evaluates and scores the written essays, aiming to provide computational models to grade essays either automatically or with minimal human involvement. While there are several AES studies in a variety of languages, few of them are focused on the Portuguese language. The main reason is the lack of a corpus with manually graded essays. In order to bridge this gap, we create a large corpus with several essays written by Brazilian high school students on an online platform. All of the essays are argumentative and were scored …
We develop models to classify desirable evidence and desirable reasoning revisions in student argumentative writing. We explore two ways to improve classifier performance - using the essay context of the revision, and using the feedback students received before the revision. We perform both intrinsic and extrinsic evaluation for each of our models and report a qualitative analysis. Our results show that while a model using feedback information improves over a baseline model, models utilizing context - either alone or with feedback - are the most successful in identifying desirable revisions.
