大语言模型辅助软件开发中的认知偏差
Cognitive Biases in LLM-Assisted Software Development
AI总结：
本文研究了大语言模型辅助软件开发中的认知偏差，通过混合方法分析发现LLM相关行动更易产生新型偏差，并提出15种偏差类别及缓解建议。
摘要：
大语言模型（LLMs）在软件开发中的广泛应用正在将编程从生成解决方案的活动转变为评估解决方案的活动。这种转变为新的认知挑战打开了路径，这些挑战可能放大现有的决策偏差或创造出全新的偏差。其中一种挑战源于认知偏差，即导致人们远离逻辑推理的思维模式，从而导致次优决策。本文首次全面研究了大语言模型辅助开发中的认知偏差。我们采用混合方法，结合对14名学生和专业开发者的观察研究，随后对22名额外开发者进行调查。我们对影响开发者的偏差类别与传统非LLM工作流程进行定性比较。我们的发现表明，与LLM相关的行动更可能与新型偏差相关。通过系统分析90种特定于开发者-LLM交互的认知偏差，我们开发了由认知心理学家验证的15种偏差类别。我们发现，总程序员行动中48.8%存在偏差，而开发者-LLM交互占这些偏差行动的56.4%。我们讨论了这些偏差类别的表现，提出了开发者的工具和实践，以及为LLM工具构建者提供的建议，以帮助缓解人类-人工智能编程中的认知偏差。
英文摘要：
The widespread adoption of Large Language Models (LLMs) in software development is transforming programming from a solution-generative to a solution-evaluative activity. This shift opens a pathway for new cognitive challenges that amplify existing decision-making biases or create entirely novel ones. One such type of challenge stems from cognitive biases, which are thinking patterns that lead people away from logical reasoning and result in sub-optimal decisions. How do cognitive biases manifest and impact decision-making in emerging AI-collaborative development? This paper presents the first comprehensive study of cognitive biases in LLM-assisted development. We employ a mixed-methods approach, combining observational studies with 14 student and professional developers, followed by surveys with 22 additional developers. We qualitatively compare categories of biases affecting developers against the traditional non-LLM workflows. Our findings suggest that LLM-related actions are more likely to be associated with novel biases. Through a systematic analysis of 90 cognitive biases specific to developer-LLM interactions, we develop a taxonomy of 15 bias categories validated by cognitive psychologists. We found that 48.8% of total programmer actions are biased, and developer-LLM interactions account for 56.4% of these biased actions. We discuss how these bias categories manifest, present tools and practices for developers, and recommendations for LLM tool builders to help mitigate cognitive biases in human-AI programming.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
