通过任务描述锚定代码可理解性评估
Anchoring Code Understandability Evaluations Through Task Descriptions
AI总结：
本文通过256名参与者的随机对照实验，证明代码理解任务描述中的难度提示会锚定参与者对代码可理解性的主观评分，并建议采用更稳健的测量方法以提升实验效度。
摘要：
在代码理解实验中，参与者通常会在开始时被告知预期进行何种代码理解任务。描述实验场景和实验任务会以有时难以预测和控制的方式影响参与者。特别是，描述甚至提及代码理解任务的难度可能会锚定参与者及其对任务本身的感知。在本研究中，我们在一项包含256名参与者（50名软件专业人员和206名计算机科学专业学生）的随机对照实验中，考察了任务描述中关于待理解代码难度的提示是否会在参与者自身的代码可理解性评分中产生锚定效应。主观代码评估是代码理解研究中衡量开发人员对代码理解程度的常用方法。因此，理解这些测量方法对锚定效应等认知偏差的稳健性非常重要。我们的结果表明，参与者在评估代码可理解性时受到初始场景描述的显著影响。初始提示代码难以理解会导致参与者对代码的理解难度评估高于未收到提示或收到代码易于理解提示的参与者。这一影响同时作用于学生和专业开发人员。我们讨论了在代码理解实验实施过程中可能诱发锚定效应的设计决策和情境因素示例，并建议在代码理解研究中使用更稳健的理解测量方法，以增强结果的有效性。
英文摘要：
In code comprehension experiments, participants are usually told at the beginning what kind of code comprehension task to expect. Describing experiment scenarios and experimental tasks will influence participants in ways that are sometimes hard to predict and control. In particular, describing or even mentioning the difficulty of a code comprehension task might anchor participants and their perception of the task itself. In this study, we investigated in a randomized, controlled experiment with 256 participants (50 software professionals and 206 computer science students) whether a hint about the difficulty of the code to be understood in a task description anchors participants in their own code comprehensibility ratings. Subjective code evaluations are a commonly used measure for how well a developer in a code comprehension study understood code. Accordingly, it is important to understand how robust these measures are to cognitive biases such as the anchoring effect. Our results show that participants are significantly influenced by the initial scenario description in their assessment of code comprehensibility. An initial hint of hard to understand code leads participants to assess the code as harder to understand than participants who received no hint or a hint of easy to understand code. This affects students and professionals alike. We discuss examples of design decisions and contextual factors in the conduct of code comprehension experiments that can induce an anchoring effect, and recommend the use of more robust comprehension measures in code comprehension studies to enhance the validity of results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
