通过插桩和程序状态对齐增强长代码翻译中的LLM
Enhancing LLMs in Long Code Translation through Instrumentation and Program State Alignment
AI总结:
针对LLM在长代码翻译中因长度和复杂语义导致性能下降的问题,提出大规模基准LongTrans进行评估,并设计PAST方法通过插桩捕获并对齐程序运行时状态,利用输出追踪纠正错误,显著提升了多个模型的翻译准确率。
摘要:
代码翻译旨在不同编程语言之间转换代码并保持功能不变,应用于跨平台开发和软件迁移。大型语言模型(LLMs)的最新进展改善了代码翻译,但挑战依然存在,特别是在推断程序功能方面。随着代码变得更长更复杂,这些问题更加严重,当前LLMs难以处理长度和复杂语义。为评估LLMs在长代码翻译上的表现,我们引入了LongTrans,这是一个大规模基于执行的基准,包含C++、Java和Python程序,长度从数百到数千个token不等。我们对12个LLMs的实证研究显示,随着代码长度增加,性能急剧下降,即使表现最好的模型GPT-4o也仅达到57.51%的计算准确率。这凸显了在长代码翻译方面进一步研究的必要性。我们认为,代码翻译应在跨语言转换语法和关键字的同时保持功能不变。尽管外观不同,程序状态在执行过程中应保持一致。为此,我们提出PAST(Program State Alignment augmented Translation,程序状态对齐增强翻译),它集成了插桩技术以在翻译过程中捕获并对齐程序状态。该方法首次利用LLMs在原始和翻译代码中插入插桩,在运行时追踪程序状态。通过提示LLM根据输出追踪纠正错误,我们缓解了不一致性并提高了翻译准确率。实验结果显示显著改进:GPT-4o的计算准确率从57.51%提升至84.70%,Mistral-Large-2从50.68%提升至69.97%,DeepSeek-Coder-V2从52.45%提升至76.43%。这些改进在不同模型和数据集上保持一致,消融研究证实了插桩和状态对齐的益处。
英文摘要:
Code translation aims to transform code between programming languages while preserving functionality, with applications in cross-platform development and software migration. Recent advances in Large Language Models (LLMs) have improved code translation, but challenges remain, particularly in inferring program functionality. These issues worsen with longer and more complex code, where current LLMs struggle to handle length and intricate semantics. To evaluate LLMs on long code translation, we introduce LongTrans, a large-scale execution-based benchmark with C++, Java, and Python programs, ranging from hundreds to thousands of tokens. Our empirical study of 12 LLMs reveals a sharp performance decline as code length increases, with even the best-performing model, GPT-4o, achieving only 57.51% computational accuracy. This highlights the need for further research in long code translation. We argue that code translation should maintain invariant functionality while transforming syntax and keywords across languages. Despite differences in appearance, program states should remain consistent throughout execution. To address this, we propose PAST (Program State Alignment augmented Translation), which integrates instrumentation to capture and align program states during translation. This approach is the first to leverage LLMs to insert instrumentation in both original and translated code, tracing program states at runtime. By prompting the LLM to correct errors based on output traces, we mitigate inconsistencies and enhance translation accuracy. Experimental results show significant improvements, with computational accuracy rising from 57.51% to 84.70% for GPT-4o, 50.68% to 69.97% for Mistral-Large-2, and 52.45% to 76.43% for DeepSeek-Coder-V2. These improvements are consistent across models and datasets, with ablation studies confirming the benefits of instrumentation and state alignment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。