arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.13586cs.CLcs.AI
收藏

平衡 LLM 采样中的多样性与风险:如何为开放式文本生成选择方法和参数

Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation

University of Mannheim(曼海姆大学);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全研究中心)

Yuxuan Zhou, Margret Keuper, Mario Fritz

AI总结:

针对开放式 LLM 生成中截断采样方法及参数选择依赖经验的问题,本文基于保留完整句上下文的前缀树,逐解码步评估多样性与风险权衡,系统比较现有方法并提供参数选择指南。

摘要:

基于采样的解码策略已在大量应用中被大型语言模型(LLM)广泛采用,旨在通过温度调节和尾部截断来平衡多样性与质量。考虑到候选下一个 token 对不同前缀具有很强的依赖性,近期研究提出自适应地截断 LLM 预测分布的尾部。尽管这些方法在开放式文本生成任务上报告了更好的结果,但其效果高度依赖于精心设置的参数和有限的示例文本。本文提出一种系统性方法,用于评估截断采样方法的能力;该方法基于我们收集的、可保留完整句子上下文的前缀树,在每个解码步骤考虑多样性与风险之间的权衡。我们的工作对现有截断采样方法进行了全面比较,并可作为用户进行参数选择的实用指南。

英文摘要:

Sampling-based decoding strategies have been widely adopted for Large Language Models (LLMs) in numerous applications, targeting a balance between diversity and quality via temperature tuning and tail truncation. Considering the strong dependency of the candidate next tokens on different prefixes, recent studies propose to adaptively truncate the tail of LLMs' predicted distribution. Although improved results have been reported with these methods on open-ended text generation tasks, the results are highly dependent on the curated parameters and the limited exemplar text. In this paper, we propose a systematic way to estimate the capacity of a truncation sampling method by considering the trade-off between diversity and risk at each decoding step, based on our collected prefix tree which preserves the context of a full sentence. Our work offers a comprehensive comparison of existing truncation sampling methods and serves as a practical user guideline for their parameter selection.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑