CliqueParcel：一种联合优化效率与忠实度的LLM提示批处理方法
CliqueParcel: An Approach For Batching LLM Prompts That Jointly Optimizes Efficiency And Faithfulness
AI总结：
本文提出CliqueParcel提示批处理方法，通过重新定义效率度量并设计多种批处理子方法，在提升LLM推理效率的同时保持输出忠实度，并在八类数据集上验证了其有效性。
摘要：
大语言模型（LLMs）在近期研究中已变得至关重要。然而，在推理过程中，LLMs仍然需要大量资源。本文提出CliqueParcel，一种旨在通过提示批处理提升LLMs效率的方法。现有的推理效率优化策略往往以牺牲输出质量为代价，导致“折扣输出”问题。该问题可能导致准确率下降或输出不够详细。CliqueParcel正是我们针对这一挑战给出的答案。在确保准确率并最小化与原始输出的偏差（即忠实度）的同时，我们的方法显著提升了推理效率。为奠定基础，我们首先重新定义了效率度量方式，排除了因输出长度缩短而带来的运行时间减少。随后，我们给出了效率与忠实度之间的综合权衡分析，以阐明“折扣输出”问题的本质。在CliqueParcel框架内，我们提出了多种批处理子方法，并讨论了它们各自适用的具体场景。在评估阶段，CliqueParcel在八个广泛认可的数据集上进行了测试，这些数据集可分为三类：阅读理解、开源问答和推理。我们的实验探究了CliqueParcel的性能表现，包括效率、忠实度以及二者之间的权衡。本研究为推理效率提供了新的见解，并展示了具有前景的性能。
英文摘要：
Large language models (LLMs) have become pivotal in recent research. However, during the inference process, LLMs still require substantial resources. In this paper, we propose CliqueParcel, a method designed to improve the efficiency of LLMs via prompt batching. Existing strategies to optimize inference efficiency often compromise on output quality, leading to a discounted output problem. This issue might result in reduced accuracy or outputs that are less detailed. CliqueParcel is our answer to this challenge. While ensuring accuracy and minimizing deviations from the original outputs (i.e., faithfulness), our method significantly improves efficiency during inference. To lay the groundwork, we first redefine efficiency measurements by excluding the reduction in running time due to shorter lengths. Then, we provide a comprehensive trade-off between efficiency and faithfulness to clarify the nature of the 'discounted output' problem. Within the CliqueParcel framework, we suggest multiple batching sub-methods and discuss the specific scenarios in which they can be applied. During evaluation, CliqueParcel is tested on eight widely recognized datasets, which can be classified into three types: reading comprehension, open-source question-answering, and reasoning. Our experiments explore the performance of CliqueParcel, including efficiency, faithfulness, and the trade-off between them. This work provides novel insights into inference efficiency and demonstrates promising performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
