从绝对代码可理解性预测到相对代码可理解性预测
From Absolute to Relative Code Comprehensibility Prediction
AI总结：
该研究针对现有绝对代码可理解性预测模型表现不佳的问题，提出相对可理解性预测任务，通过实验证明其在多数配置下优于基线模型，且受从业者青睐。
摘要：
自动预测代码可理解性可支持重构、代码审查等任务。现有指标与人类理解能力的相关性较差，因此催生了直接从代码和开发者特征预测可理解性的机器学习模型。现有模型预测绝对可理解性（AC），即单个代码片段的可理解性数值，但由于AC是复杂认知过程的主观替代指标，这类模型表现不佳。我们提出相对可理解性（RC）作为替代任务：给定两个代码片段，预测哪一个更易理解或二者是否相当。我们假设RC更易于学习，因为它仅需识别代码片段间的区分特征，而非估计绝对数值。我们使用来自两项现有研究的150个Java代码片段和12540个人类可理解性测量数据，在经典机器学习模型、卷积神经网络（CNN）和两个大型语言模型（LLM）上比较AC和RC预测，评估代码片段级（聚合）和开发者级（个体判断）的预测效果。AC模型极少优于简单基线（平均相对提升最多为33.4%），而代码片段级RC模型在96.8%的配置中优于基线，在不同架构上的提升最高达159.8%，不过开发者级结果波动更大。我们对38名从业者进行调查，发现AC和RC均有用，在重构、审查等对比导向任务中，从业者对RC的偏好更强。
英文摘要：
Automatically predicting code comprehensibility could support tasks such as refactoring and code review. Existing metrics correlate poorly with human comprehension, motivating ML models that predict comprehensibility directly from code and developer features. Prior models predict absolute comprehensibility (AC), a comprehensibility value for an isolated snippet, but perform poorly since AC is a subjective proxy for a complex cognitive process. We propose relative comprehensibility (RC) as an alternative task: given two snippets, predict which is easier to understand or whether they are comparable. We hypothesize RC is easier to learn, since it only requires identifying distinguishing features between snippets rather than estimating absolute values. Using 150 Java snippets and 12,540 human comprehensibility measurements from two prior studies, we compare AC and RC prediction across classical ML models, a CNN, and two LLMs, evaluating both snippet-wise (aggregate) and developer-wise (individual-judgment) predictions. AC models rarely beat simple baselines (at most 33.4\% average relative improvement), while snippet-wise RC models outperform baselines in 96.8\% of configurations, with gains up to 159.8\% consistent across architectures, though developer-wise results are more variable. We surveyed 38 practitioners and found both AC and RC useful, with a stronger preference for RC in comparison-oriented tasks like refactoring and review.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
