R-U-SURE？通过在随机用户意图间最大化效用实现具备不确定性感知的代码建议
R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents
AI总结：
针对大语言模型代码生成易出错的问题，提出R-U-SURE方法，结合多种解码与图技术，基于生成模型样本构建不确定性感知代码建议，无需重训练适配多交互模式，估计准确性优于基线，已开源。
摘要：
大语言模型在预测代码等结构化文本方面展现出出色效果，但其输出中也常出现错误和幻觉。当用于辅助软件开发人员时，这些模型可能会产生用户必须回头修复的错误，更糟糕的是，可能会引入用户完全没注意到的隐蔽bug。我们提出了Randomized Utility-driven Synthesis of Uncertain REgions（R-U-SURE，随机效用驱动的不确定区域合成）方法，该方法基于目标条件效用的决策论模型构建具备不确定性感知的建议，使用生成模型的随机样本作为终端用户未被观测到的可能意图的代理。我们的技术结合了最小贝叶斯风险解码、对偶分解和决策图，仅需能对任意代码生成模型进行采样访问、并可选提供AST（抽象语法树）解析器，就能高效生成结构化的不确定性摘要。我们在三项开发者辅助任务上对R-U-SURE进行了验证，结果表明它无需重新训练模型即可应用于不同的用户交互模式，且比基于词元概率的基线方法能产生更准确的不确定性估计。我们还将实现代码作为开源库发布，地址为https://github.com/google-research/r_u_sure。
英文摘要：
Large language models show impressive results at predicting structured text such as code, but also commonly introduce errors and hallucinations in their output. When used to assist software developers, these models may make mistakes that users must go back and fix, or worse, introduce subtle bugs that users may miss entirely. We propose Randomized Utility-driven Synthesis of Uncertain REgions (R-U-SURE), an approach for building uncertainty-aware suggestions based on a decision-theoretic model of goal-conditioned utility, using random samples from a generative model as a proxy for the unobserved possible intents of the end user. Our technique combines minimum-Bayes-risk decoding, dual decomposition, and decision diagrams in order to efficiently produce structured uncertainty summaries, given only sample access to an arbitrary generative model of code and an optional AST parser. We demonstrate R-U-SURE on three developer-assistance tasks, and show that it can be applied different user interaction patterns without retraining the model and leads to more accurate uncertainty estimates than token-probability baselines. We also release our implementation as an open-source library at https://github.com/google-research/r_u_sure.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
