arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.19825cs.CLcs.AI
收藏

简洁思考:输出长度对LLM推理和成本的影响

Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost

Scuola Superiore Sant’Anna(圣安娜高等学校);National Research Council of Italy(意大利国家研究委员会);Sapienza Università di Roma(罗马第一大学);Mediavoice Srl(Mediavoice股份公司)

Sania Nayab, Giulio Rossolini, Marco Simoni, Andrea Saracino, Giorgio Buttazzo, Nicolamaria Manes, Fabrizio Giacomelli

AI总结:

本文提出CCoT方法,通过控制输出长度提升LLM推理的简洁性和效率,并验证了相关指标的有效性。

摘要:

当今大型语言模型(LLMs)能够解决具有挑战性的问题回答任务,并且提示工程技术,如思维链(CoT),因增强输出的解释性和正确性而受到关注。然而,许多模型和技术倾向于产生过于冗长和详尽的答案,导致在简洁性和生成时间上出现问题。为了解决这一问题,本文通过引入并提出新的指标来评估模型和相关提示技术的正确简洁性,分析输出长度对LLM推理管道的影响。然后,我们检查通过精细化提示工程策略,即受约束的思维链(CCoT),来控制输出长度的影响,该策略鼓励模型生成更简洁的输出。为了更好地理解此类提示的影响,我们还引入了两个额外的分数来分析简洁性,这些分数以生成答案中的冗余性和信息流为衡量标准。在预训练的LLM和多个数据集上的实验表明,所提出指标的益处以及CCoT在不同模型中的有效性。

英文摘要:

Today's large language models (LLMs) can solve challenging question-answering tasks, and prompt engineering techniques, such as chain-of-thought (CoT), have gained attention for enhancing the explanation and correctness of outputs. However, many models and techniques tend to produce excessively verbose and lengthy answers, leading to issues with both conciseness and generation time. To address this, this paper analyzes the impact of output lengths on LLM inference pipelines by introducing and proposing novel metrics to evaluate the \textit{correct conciseness} of a model and related prompting techniques. Then, we examine the impact of controlling output length through a refined prompt engineering strategy, Constrained-CoT (CCoT), which encourages the model to produce more concise outputs. To better understand the effects of such a prompt, we also introduce two additional scores for analyzing the conciseness, measured in terms of redundancy and information flow in generated answers. Experiments on pretrained LLMs and multiple datasets demonstrate the benefits of the proposed metrics and the effectiveness of CCoT across different models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑