arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.04723cs.CLcs.AI
收藏

将长上下文大语言模型研究从输入转向输出

Shifting Long-Context LLMs Research from Input to Output

Singapore University of Technology and Design(新加坡科技设计大学);Tsinghua University(清华大学)

Yuhao Wu, Yushi Bai, Zhiqing Hu, Shangqing Tu, Ming Shan Hee, Juanzi Li, Roy Ka-Wei Lee

AI总结:

本文指出当前长上下文LLM研究偏重输入理解而忽视长输出生成,呼吁转向开发能生成高质量长篇文本的基础模型,以填补小说写作、长期规划等任务中的能力缺口。

摘要:

近年来,长上下文大语言模型(LLMs)的进展主要集中在处理扩展的输入上下文上,在长上下文理解方面取得了显著进步。然而,同样关键的生成长篇输出的方面却受到的关注相对较少。本文倡导自然语言处理(NLP)研究发生范式转变,以应对长输出生成的挑战。诸如小说写作、长期规划和复杂推理等任务要求模型理解广泛的上下文,并生成连贯、上下文丰富且逻辑一致的扩展文本。这些需求凸显了当前LLM能力中的一个关键缺口。我们强调这一尚未充分探索领域的重要性,并呼吁集中力量开发专为生成高质量长篇输出而定制的基础LLMs,这对现实世界应用具有巨大潜力。

英文摘要:

Recent advancements in long-context Large Language Models (LLMs) have primarily concentrated on processing extended input contexts, resulting in significant strides in long-context comprehension. However, the equally critical aspect of generating long-form outputs has received comparatively less attention. This paper advocates for a paradigm shift in NLP research toward addressing the challenges of long-output generation. Tasks such as novel writing, long-term planning, and complex reasoning require models to understand extensive contexts and produce coherent, contextually rich, and logically consistent extended text. These demands highlight a critical gap in current LLM capabilities. We underscore the importance of this under-explored domain and call for focused efforts to develop foundational LLMs tailored for generating high-quality, long-form outputs, which hold immense potential for real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑