探索用于代码分析的自动化代码评估系统与资源：一项综合综述
Exploring Automated Code Evaluation Systems and Resources for Code Analysis: A Comprehensive Survey
AI总结：
本文综合综述自动化代码评估系统的应用分类、数据集与资源，并梳理机器学习代码任务及AOJ平台实践。
摘要：
自动化代码评估系统（AES）主要旨在可靠地评估用户提交的代码。由于其广泛的应用范围以及宝贵资源的积累，AES正变得越来越受欢迎。然而，针对AES在多样编程任务中的应用及其真实世界资源探索的研究仍然缺乏。在本研究中，我们对AES及其资源进行了全面综述。该综述探讨了AES的应用领域、可用资源以及编程任务中的资源利用情况。根据应用，AES被分为编程竞赛、编程学习与教育、招聘、在线编译器以及附加模块。我们探索了这些系统可用于研究、分析和编程任务的数据集及其他资源。此外，我们概述了机器学习驱动的编程任务，例如缺陷检测、代码审查、理解、重构、搜索、表示和修复。这些任务均使用真实数据集执行。另外，我们从系统设计（硬件和软件）、运营（竞赛和教育）以及研究的角度，简要讨论了Aizu Online Judge平台作为AES真实案例的情况。这是由于AOJ平台具备可扩展性（编程教育、竞赛和练习）、开放的内部特性（硬件和软件）、研究社区的关注、开源数据（例如解答代码和提交文档）以及透明性。我们还分析了该系统多年来的整体性能以及所感知到的挑战。
英文摘要：
The automated code evaluation system (AES) is mainly designed to reliably assess user-submitted code. Due to their extensive range of applications and the accumulation of valuable resources, AESs are becoming increasingly popular. Research on the application of AES and their real-world resource exploration for diverse coding tasks is still lacking. In this study, we conducted a comprehensive survey on AESs and their resources. This survey explores the application areas of AESs, available resources, and resource utilization for coding tasks. AESs are categorized into programming contests, programming learning and education, recruitment, online compilers, and additional modules, depending on their application. We explore the available datasets and other resources of these systems for research, analysis, and coding tasks. Moreover, we provide an overview of machine learning-driven coding tasks, such as bug detection, code review, comprehension, refactoring, search, representation, and repair. These tasks are performed using real-life datasets. In addition, we briefly discuss the Aizu Online Judge platform as a real example of an AES from the perspectives of system design (hardware and software), operation (competition and education), and research. This is due to the scalability of the AOJ platform (programming education, competitions, and practice), open internal features (hardware and software), attention from the research community, open source data (e.g., solution codes and submission documents), and transparency. We also analyze the overall performance of this system and the perceived challenges over the years.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
