面向软件元数据的生成式AI：2023年FIRE会议软件工程信息检索赛道概述
Generative AI for Software Metadata: Overview of the Information Retrieval in Software Engineering Track at FIRE 2023
AI总结：
本文概述2023年FIRE会议IRSE赛道，该赛道围绕基于人工与大模型标签的代码注释自动评估二分类任务，17支团队提交56项实验，经F1-Score等评估发现大模型生成标签会增加模型偏差但可减少过拟合。
摘要：
软件工程信息检索（Information Retrieval in Software Engineering, IRSE）赛道旨在基于人工与大语言模型生成的标签，在机器学习框架下开发代码注释自动评估的解决方案。该赛道设置了一项二分类任务，将注释划分为有用和无用两类。数据集包含从GitHub开源C语言项目中提取的9048条代码注释及其对应代码片段对，另有各参赛团队使用大语言模型单独生成的补充数据集。来自多所高校和软件公司的17支团队共提交了56项实验。评审采用F1-Score进行定量评估，同时基于所开发的特征类型、所用监督学习模型及其对应超参数开展定性评估。大语言模型生成的标签会增加预测模型的偏差，但能降低过拟合程度。
英文摘要：
The Information Retrieval in Software Engineering (IRSE) track aims to develop solutions for automated evaluation of code comments in a machine learning framework based on human and large language model generated labels. In this track, there is a binary classification task to classify comments as useful and not useful. The dataset consists of 9048 code comments and surrounding code snippet pairs extracted from open source github C based projects and an additional dataset generated individually by teams using large language models. Overall 56 experiments have been submitted by 17 teams from various universities and software companies. The submissions have been evaluated quantitatively using the F1-Score and qualitatively based on the type of features developed, the supervised learning model used and their corresponding hyper-parameters. The labels generated from large language models increase the bias in the prediction model but lead to less over-fitted results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
