针对LLM一个有趣失败的高效解决方案:长上下文窗口并不意味着LLM能完美分析长序列
Efficient Solutions For An Intriguing Failure of LLMs: Long Context Window Does Not Mean LLMs Can Analyze Long Sequences Flawlessly
AI总结:
本文揭示了LLM在处理长输入序列时存在的局限性,并使用多个数据集和模型进行评估,提出了专门的解决方案,将长序列处理性能最高提升50%,同时大幅降低API成本和延迟。
摘要:
大型语言模型(LLM)在理解和分析冗长的序列输入方面展现出了卓越的能力,这得益于其广阔的上下文窗口,允许在单次前向传递中处理数百万个token。然而,本文揭示了一个令人惊讶的局限性:LLM在处理长输入序列时表现不佳。我们使用三个数据集和两个任务(情感分析和新闻分类)在各种LLM上调查了这一问题,包括Claude 3、Gemini Pro、GPT 3.5 Turbo、Llama 3 Instruct和Mistral Instruct模型。为了解决这一局限性,我们提出并评估了专门的解决方案,将LLM在长输入序列上的性能最高提升了50%,同时分别将API成本和延迟最高降低了93%和50%。
英文摘要:
Large Language Models (LLMs) have demonstrated remarkable capabilities in comprehending and analyzing lengthy sequential inputs, owing to their extensive context windows that allow processing millions of tokens in a single forward pass. However, this paper uncovers a surprising limitation: LLMs fall short when handling long input sequences. We investigate this issue using three datasets and two tasks (sentiment analysis and news categorization) across various LLMs, including Claude 3, Gemini Pro, GPT 3.5 Turbo, Llama 3 Instruct, and Mistral Instruct models. To address this limitation, we propose and evaluate ad-hoc solutions that substantially enhance LLMs' performance on long input sequences by up to 50%, while reducing API cost and latency by up to 93% and 50%, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。