性别化提示与大语言模型代码审查:提示中的性别线索如何影响代码质量和评估
Gendered Prompting and LLM Code Review: How Gender Cues in the Prompt Shape Code Quality and Evaluation
AI总结:
研究探讨性别化提示对代码生成和审查的影响,发现性别线索在代码质量上影响有限,但对代码审查存在系统性偏见。
摘要:
大语言模型(LLMs)正越来越多地嵌入编程工作流中,从代码生成到自动代码审查。然而,性别化沟通风格如何与LLM辅助编程和代码审查互动仍缺乏研究。本文通过混合方法试点研究,探讨提示中的性别相关语言差异是否影响代码生成结果和代码审查决策。在三个互补研究中,我们分析(i)收集的现实世界编码提示,(ii)受控用户研究,开发者在LLM协助下解决相同编程任务,以及(iii)基于LLM的模拟评估框架,系统地变化性别编码提示风格和审查者人设。我们发现,性别相关的提示风格差异微妙但可测量,女性作者的提示使用更间接和深入的语言,但并未在功能正确性或静态代码质量上产生一致差距。对于LLM代码审查,我们观察到系统性偏见:平均而言,模型更倾向于批准女性作者的代码,尽管质量相当。受控实验显示,性别编码提示风格影响代码长度和可维护性,而审查者行为因模型而异。我们的发现表明,LLM辅助编程中的公平性风险更多源于LLM评估而非生成准确性,因为LLM越来越多地被部署为自动代码审查工具。
英文摘要:
LLMs are increasingly embedded in programming workflows, from code generation to automated code review. Yet, how gendered communication styles interact with LLM-assisted programming and code review remains underexplored. We present a mixed-methods pilot study examining whether gender-related linguistic differences in prompts influence code generation outcomes and code review decisions. Across three complementary studies, we analyze (i) collected real-world coding prompts, (ii) a controlled user study, in which developers solve identical programming tasks with LLM assistance, and (iii) an LLM-based simulated evaluation framework that systematically varies gender-coded prompt styles and reviewer personas. We find that gender-related differences in prompting style are subtle but measurable, with female-authored prompts exhibiting more indirect and involved language, which does not translate into consistent gaps in functional correctness or static code quality. For LLM code review, in contrast, we observe systematic biases: on average, models approve female-authored code more, despite comparable quality. Controlled experiments show that gender-coded prompt style affect code length and maintainability, while reviewer behavior varies across models. Our findings suggest that fairness risks in LLM-assisted programming arise less from generation accuracy than from LLM evaluation, as LLMs are increasingly deployed as automated code reviewers.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。