arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.23933cs.CL
收藏

大语言模型可通过自扩展生成长文本

Language Models can Self-Lengthen to Generate Long Texts

Qwen Team, Alibaba Inc.(通义千问团队,阿里巴巴集团)

Shanghaoran Quan, Tianyi Tang, Bowen Yu, An Yang, Dayiheng Liu, Bofei Gao, Jianhong Tu, Yichang Zhang, Jingren Zhou, Junyang Lin

AI总结:

针对大语言模型长文本生成能力不足的问题,提出仅依赖模型自身知识的Self-Lengthen迭代训练框架,通过生成与扩展双角色迭代训练提升长文本生成效果,在主流开源模型上表现优于现有方法。

摘要:

大语言模型(LLMs)近期的进展显著提升了其处理长上下文的能力,但在生成连贯一致的长输出方面仍存在明显差距。这一局限源于训练缺口:预训练阶段缺乏有效的长文本生成指令,而后期训练数据主要由短查询-响应对构成。当前的指令回译、行为模仿等方法面临数据质量、版权问题以及专有模型使用限制等挑战。本文提出了一种名为Self-Lengthen(自扩展)的创新迭代训练框架,该框架仅利用大语言模型的内在知识与技能,无需辅助数据或专有模型。框架包含生成器(Generator)和扩展器(Extender)两个角色:生成器生成初始响应,再由扩展器将其拆分并扩展,得到更长的新响应,用于迭代训练生成器和扩展器。通过这一过程,模型逐步被训练以处理越来越长的响应。在基准测试和人工评估中的实验表明,将Self-Lengthen应用于Qwen2、LLaMA3等顶级开源大语言模型时,其长文本生成效果优于现有方法。我们的代码已在https://github.com/QwenLM/Self-Length公开。

英文摘要:

Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to process long contexts, yet a notable gap remains in generating long, aligned outputs. This limitation stems from a training gap where pre-training lacks effective instructions for long-text generation, and post-training data primarily consists of short query-response pairs. Current approaches, such as instruction backtranslation and behavior imitation, face challenges including data quality, copyright issues, and constraints on proprietary model usage. In this paper, we introduce an innovative iterative training framework called Self-Lengthen that leverages only the intrinsic knowledge and skills of LLMs without the need for auxiliary data or proprietary models. The framework consists of two roles: the Generator and the Extender. The Generator produces the initial response, which is then split and expanded by the Extender. This process results in a new, longer response, which is used to train both the Generator and the Extender iteratively. Through this process, the models are progressively trained to handle increasingly longer responses. Experiments on benchmarks and human evaluations show that Self-Lengthen outperforms existing methods in long-text generation, when applied to top open-source LLMs such as Qwen2 and LLaMA3. Our code is publicly available at https://github.com/QwenLM/Self-Lengthen.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑