基于代码图的未见代码库数据合成与训练
Unseen-Codebases-Domain Data Synthesis and Training Based on Code Graphs
AI总结：
提出UCD-Training框架，通过代码图构建和两阶段训练，提升模型对未见代码库的推理能力和数据合成效果。
摘要：
在新的软件框架发布背景下，大型语言模型（LLMs）往往表现出性能不佳和高幻觉率，因为它们在训练过程中没有接触过这样的环境。尽管推理时的增强技术如检索增强生成（RAG）可以部分缓解幻觉，但仅通过提示进行知识注入是不够的，无法使模型充分理解代码库中不同组件之间的内在关系，或进行正确的组合推理和应用。虽然通过训练后可以实现显式知识注入，但与公共代码领域相比，未见代码库通常只提供源代码，缺乏大量高质量、面向使用的代码，这些代码可以直接作为训练数据。因此，现有数据合成方法在仅限于源代码的情况下，无法充分捕捉未见代码库的使用场景。为了解决这些挑战，我们提出了UCD-Training，一种基于从未见代码库构建的代码图的面向推理的数据合成两阶段训练框架。UCD-Training首先解析源代码以构建代码图，然后使用文件级依赖数据进行依赖保持的持续预训练（CPT），接着在三种合成数据上进行图基础的监督微调（SFT），这些合成数据通过显式推理轨迹增强：（1）单跳关系推理数据，（2）组合API推理数据，（3）代码库使用数据。我们进一步引入了一个新的基准测试集UnseenCodeBench，用于在未见代码库上进行代码生成，并在多个代码库上进行了全面的实验。
英文摘要：
In the context of newly release software frameworks, large language models (LLMs) often exhibit poor performance and a high rate of hallucination, as they are not exposed to such environments during training. Although inference-time augmentation techniques such as retrieval-augmented generation (RAG) can partially mitigate hallucinations, knowledge injection through prompting alone is insufficient to enable models to fully understand the intrinsic relationships among different components of a codebase, or to reason about the correct compositions and apply. Although explicit knowledge injection can be achieved through post-training, compared with public code domains, unseen codebases typically provide only source code and lack large volumes of high-quality, usage-oriented code that can be directly leveraged as training data. Consequently, existing data synthesis approaches are insufficient to adequately capture unseen codebases usage scenarios when restricted to source code alone. To address these challenges, we propose UCD-Training, a two-stage training framework for reasoning-aware data synthesis grounded in a code graph constructed from unseen codebases. UCD-Training first parses the source code to build a code graph, then conducts dependency-preserving continued pretraining (CPT) using file-level dependency data, followed by graph-grounded supervised fine-tuning (SFT) on three types of synthesized data augmented with explicit reasoning traces: (1) single-hop relation reasoning data, (2) compositional API reasoning data, and (3) codebase utilization data. We further introduce a new benchmark, UnseenCodeBench, for code generation on unseen codebases and conduct comprehensive experiments across multiple codebases.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
