程序合成器的生成式解释
Generative Explanations for Program Synthesizers
AI总结：
针对程序合成器是黑盒、现有LLM解释合成代码准确率低的问题，提出结合合成算法细粒度数据优化提示、用验证器与第二LLM校验的生成解释方法，将名称准确率从24%提至79%，用户认可度远高于基线。
摘要：
尽管程序合成技术已取得重大进展，但它们仍是算法黑盒。虽然这类技术能保证合成成功时生成的实现满足规范，但不会提供关于该实现如何工作、规范以何种方式被实现的额外信息。解答这些问题的一种可能思路是使用大语言模型（LLM）构建人类可读的解释。但遗憾的是，实验表明，当LLM被用于处理程序合成器生成的非惯用代码时，经常会产生无意义或具有误导性的解释。\n本文提出了一种可靠地为合成实现添加解释性名称的方法。我们从合成算法中恢复细粒度的输入输出数据，以优化提供给LLM的提示词，并且在将生成的解释呈现给用户之前，结合程序验证器和第二个语言模型对其进行验证。这些技术共同大幅提升了生成名称的准确率，从24%提升至79%。通过两项小型用户研究我们发现，用户明显更偏好我们的技术生成的解释——76%的反馈认为所呈现的名称合适，而基线方法仅有2%的反馈认可其建议，且这些生成的名称确实能帮助用户理解合成得到的实现。
英文摘要：
Despite great advances in program synthesis techniques, they remain algorithmic black boxes. Although they guarantee that when synthesis is successful, the implementation satisfies the specification, they provide no additional information regarding how the implementation works or the manner in which the specification is realized. One possibility to answer these questions is to use large language models (LLMs) to construct human-readable explanations. Unfortunately, experiments reveal that LLMs frequently produce nonsensical or misleading explanations when applied to the unidiomatic code produced by program synthesizers. In this paper, we develop an approach to reliably augment the implementation with explanatory names. We recover fine-grained input-output data from the synthesis algorithm to enhance the prompt supplied to the LLM, and use a combination of a program verifier and a second language model to validate the proposed explanations before presenting them to the user. Together, these techniques massively improve the accuracy of the proposed names, from 24% to 79% respectively. Through a pair of small user studies, we find that users significantly prefer the explanations produced by our technique (76% of responses indicating the appropriateness of the presenting names) to the baseline (with only 2% of responses approving of the suggestions), and that the proposed names measurably help users in understanding the synthesized implementation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
