Citekit：面向大语言模型引文生成的模块化工具包
Citekit: A Modular Toolkit for Large Language Model Citation Generation
AI总结：
本文提出Citekit，一个模块化开源工具包，用于标准化和评估LLM引文生成方法，并通过实验分析提出self-RAG snippet方法以平衡答案准确性与引文质量。
摘要：
使大语言模型（LLMs）在问答（QA）任务中生成引文是一种新兴范式，旨在当LLMs利用外部参考文献生成答案时，增强其响应的可验证性。然而，目前尚无统一框架来标准化和公平比较不同的引文生成方法，导致不同方法的复现和全面评估存在困难。为解决上述问题，我们推出了\name，一个开源且模块化的工具包，旨在促进现有引文生成方法的实现与评估，同时推动开发新方法以提升LLM输出中的引文质量。该工具高度可扩展，允许用户利用4个主要模块和14个组件构建流水线，以评估现有方法或创新设计。我们使用两个最先进的LLM和11个引文生成基线进行的实验表明，不同模块在答案准确性和引文质量提升方面各有优势，同时揭示了提升细粒度所面临的挑战。基于对组件有效性的分析，我们提出了一种新方法self-RAG \snippet，在答案准确性和引文质量之间取得了平衡。Citekit已在https://github.com/SjJ1017/Citekit发布。
英文摘要：
Enabling Large Language Models (LLMs) to generate citations in Question-Answering (QA) tasks is an emerging paradigm aimed at enhancing the verifiability of their responses when LLMs are utilizing external references to generate an answer. However, there is currently no unified framework to standardize and fairly compare different citation generation methods, leading to difficulties in reproducing different methods and a comprehensive assessment. To cope with the problems above, we introduce \name, an open-source and modular toolkit designed to facilitate the implementation and evaluation of existing citation generation methods, while also fostering the development of new approaches to improve citation quality in LLM outputs. This tool is highly extensible, allowing users to utilize 4 main modules and 14 components to construct a pipeline, evaluating an existing method or innovative designs. Our experiments with two state-of-the-art LLMs and 11 citation generation baselines demonstrate varying strengths of different modules in answer accuracy and citation quality improvement, as well as the challenge of enhancing granularity. Based on our analysis of the effectiveness of components, we propose a new method, self-RAG \snippet, obtaining a balanced answer accuracy and citation quality. Citekit is released at https://github.com/SjJ1017/Citekit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
