arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.07425cs.SE
收藏

通过精准上下文信息注入增强LLM生成更具仓库感知能力的单元测试的能力

Enhancing LLM's Ability to Generate More Repository-Aware Unit Tests Through Precise Contextual Information Injection

Xin Yin, Chao Ni, Xinrui Li, Liushan Chen, Guojun Ma, Xiaohu Yang

AI总结:

针对大语言模型生成单元测试时因缺乏项目全局上下文易产生幻觉的问题,提出RATester,通过集成gopls语言服务器按需注入全局上下文信息,提升单元测试的仓库感知能力并减少幻觉。

摘要:

尽管已有许多基于学习的方法被提出用于单元测试生成并取得了显著性能,但它们仍存在依赖特定任务数据集的局限性。近年来,在提示工程引导下的大语言模型(LLM)因其能够处理包括单元测试生成在内的广泛任务而受到关注。尽管取得了成功,LLM在为核心方法或函数生成单元测试时可能会出现幻觉,因为它们缺乏对项目全局上下文的感知。这些幻觉可能表现为调用不存在的方法,以及错误的参数或返回值,例如参数类型或数量不匹配。虽然许多研究已经探讨了上下文的作用,但它们通常为不同的模型和核心方法提取固定模式的上下文,这可能并不适用于所有生成过程(例如,过多的无关上下文可能导致冗余,使模型无法专注于关键信息)。为了克服这一局限性,我们提出了RATester,它通过注入全局上下文信息来增强LLM生成更具仓库感知能力的单元测试的能力。为了让LLM具备类似人类测试人员的全局知识,我们集成了语言服务器gopls,它提供定义查找等基本功能来辅助LLM。当RATester遇到不熟悉的标识符(例如不熟悉的结构体名称)时,它首先利用gopls获取相关定义和文档注释,然后使用这些全局知识来引导LLM。通过利用gopls,RATester丰富了LLM对项目全局上下文的认知,从而减少了单元测试生成过程中的幻觉。

英文摘要:

Though many learning-based approaches have been proposed for unit test generation and achieved remarkable performance, they still have limitations in relying on task-specific datasets. Recently, Large Language Models (LLMs) guided by prompt engineering have gained attention for their ability to handle a broad range of tasks, including unit test generation. Despite their success, LLMs may exhibit hallucinations when generating unit tests for focal methods or functions due to their lack of awareness regarding the project's global context. These hallucinations may manifest as calls to non-existent methods, as well as incorrect parameters or return values, such as mismatched parameter types or numbers. While many studies have explored the role of context, they often extract fixed patterns of context for different models and focal methods, which may not be suitable for all generation processes (e.g., excessive irrelevant context could lead to redundancy, preventing the model from focusing on essential information). To overcome this limitation, we propose RATester, which enhances the LLM's ability to generate more repository-aware unit tests through global contextual information injection. To equip LLMs with global knowledge similar to that of human testers, we integrate the language server gopls, which provides essential features (e.g., definition lookup) to assist the LLM. When RATester encounters an unfamiliar identifier (e.g., an unfamiliar struct name), it first leverages gopls to fetch relevant definitions and documentation comments, and then uses this global knowledge to guide the LLM. By utilizing gopls, RATester enriches the LLM's knowledge of the project's global context, thereby reducing hallucinations during unit test generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑