arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.02567cs.SEcs.AI
收藏

基于LLM的软件工程中的反馈循环与代码扰动:一种C到Rust翻译系统的案例研究

Feedback Loops and Code Perturbations in LLM-based Software Engineering: A Case Study on a C-to-Rust Translation System

Otto von Guericke University(奥托·冯·格里克大学);Bosch Research(博世研究)

Martin Weiss, Jesko Hecking-Harbusch, Jochen Quante, Matthias Woehrle

AI总结:

本文研究了反馈循环、LLM选择和代码扰动对C到Rust翻译系统性能的影响,发现反馈循环可减少模型差异并提升系统鲁棒性。

摘要:

强大生成式AI的出现对各种软件工程任务,如代码修复、测试生成或语言翻译,产生了相当大的影响。尽管像GitHub Copilot这样的工具已经在交互式环境中广泛使用,但自动化方法在工业实践中使用之前需要更高的可靠性。在本文中,我们关注三个直接影响结果质量的方面:a) 自动反馈循环的影响,b) 大型语言模型(LLM)的选择,以及c) 保持行为的代码更改的影响。我们研究这些三个变量对自动化C到Rust翻译系统的影响。由于Rust的安全保证,C到Rust的翻译在工业中是一个有吸引力的用例。翻译系统基于生成和检查模式,在其中由LLM生成的Rust代码会自动检查可编译性和与原始C代码的行为等价性。对于负检查结果,LLM会在反馈循环中被重新提示以修复其输出。这些检查还允许我们评估和比较翻译系统在变化三个变量时的成功率。我们的结果表明,在没有反馈循环的情况下,LLM的选择对翻译成功率有重大影响。然而,当翻译系统使用反馈循环时,不同模型之间的差异减小。我们观察到这一点,不仅在系统的平均性能方面,而且在代码扰动下的鲁棒性方面。最后,我们还发现,由代码扰动提供的多样性甚至可以导致系统性能的提高。

英文摘要:

The advent of strong generative AI has a considerable impact on various software engineering tasks such as code repair, test generation, or language translation. While tools like GitHub Copilot are already in widespread use in interactive settings, automated approaches require a higher level of reliability before being usable in industrial practice. In this paper, we focus on three aspects that directly influence the quality of the results: a) the effect of automated feedback loops, b) the choice of Large Language Model (LLM), and c) the influence of behavior-preserving code changes. We study the effect of these three variables on an automated C-to-Rust translation system. Code translation from C to Rust is an attractive use case in industry due to Rust's safety guarantees. The translation system is based on a generate-and-check pattern, in which Rust code generated by the LLM is automatically checked for compilability and behavioral equivalence with the original C code. For negative checking results, the LLM is re-prompted in a feedback loop to repair its output. These checks also allow us to evaluate and compare the respective success rates of the translation system when varying the three variables. Our results show that without feedback loops LLM selection has a large effect on translation success. However, when the translation system uses feedback loops the differences across models diminish. We observe this for the average performance of the system as well as its robustness under code perturbations. Finally, we also identify that diversity provided by code perturbations can even result in improved system performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑