DPO-F+:使代码修复反馈与开发者偏好对齐
DPO-F+: Aligning Code Repair Feedback with Developers' Preferences
AI总结:
研究针对大语言模型在代码修复中开发者难解读输出的问题,提出DPO-f+框架,通过定义指标、构建数据集、微调模型及评估反馈质量,提升反馈准确性与对齐度,增强了代码理解和人机协作。
摘要:
大语言模型(LLMs)在软件工程任务尤其是代码修复中使用日益广泛。但开发者常难以解读模型输出,限制了人机协作。以往工作多优化生成代码,对支持理解和迭代改进的自然语言反馈关注较少。本文提出DPO-f+框架,使其与不同开发者群体需求对齐。该框架定义七个固定维度的反馈对齐指标,自动构建成对偏好数据集,用增强奖励模型的直接偏好优化(DPO)微调模型,并提供评估反馈质量的自动化协议。实验表明,DPO-f+在反馈准确性和整体对齐方面优于基线和标准DPO。在新手编程任务和SWE-Bench上,其性能均有提升,还提高了LLM判断和人工研究中的反馈对齐度。DPO-f+将LLM辅助从一次性输出转变为协作式理解流程,增强了软件工程中的代码理解和人机协作。
英文摘要:
Large Language Models (LLMs) are increasingly used in software engineering tasks, especially code repair. However, developers often struggle to interpret model outputs, limiting effective human--AI teaming, where humans and AI work toward a shared objective. Prior work mainly optimizes generated code, giving less attention to natural-language feedback that supports comprehension and iterative improvement. We present \textsc{DPO-f+}, a framework that aligns code-repair feedback with the needs of different developer groups, including novices and proficient developers. It (1) defines feedback-alignment metrics across seven fixed dimensions with task-specific descriptions; (2) automatically constructs pairwise preference datasets from code-repair tasks; (3) fine-tunes models using Direct Preference Optimization (DPO) augmented with a reward model; and (4) provides an automated protocol for evaluating feedback quality. Empirically, \textsc{DPO-f+} outperforms both the baseline and standard DPO in feedback accuracy and overall alignment. On novice programming tasks, \textsc{DPO-f+} improves Pass@1 by 5.71 percentage points (pp) over the baseline and 3.30 pp over DPO. On \textsc{SWE-Bench}, it improves issue-resolution rate by 1.67 pp over DPO and 4.67 pp over the baseline. It also improves feedback alignment by both LLM judges and a human study with 200 developers: beginners preferred \textsc{DPO-f+} in 71.5% of comparisons, with overall preference above chance ($p=0.0057$). By better aligning feedback with developer needs, \textsc{DPO-f+} turns LLM assistance from a one-shot output into a collaborative sense-making workflow, enhancing code comprehension and human--AI teaming in software engineering.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。