arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.22608cs.AIcs.CL
收藏

理解多实例处理中LLM性能下降:实例数量和上下文长度的作用

Understanding LLM Performance Degradation in Multi-Instance Processing: The Roles of Instance Count and Context Length

School of Computer Science and Informatics, Cardiff University(卡迪夫大学计算机科学与信息学学院)

Jingxuan Chen, Mohammad Taher Pilehvar, Jose Camacho-Collados

AI总结:

研究多实例处理中LLM性能下降的原因,发现实例数量和上下文长度均影响性能,但实例数量影响更显著。

摘要:

用户常依赖大语言模型(LLM)处理多个文档或分析多个实例。例如,分析多部电影评论的整体情绪需要LLM分别处理每条评论以提供最终汇总答案。尽管LLM在单任务上的性能通常较高,但关于多实例输入时LLM表现的研究较少。本文对LLM在擅长单任务中的多实例处理(MIP)能力进行了全面评估。结果表明,所有LLM在实例数量较小时(约20-100)表现出轻微性能下降,随后在较大实例数量时出现性能崩溃。关键发现显示,虽然上下文长度与这种下降有关,但实例数量对最终结果的影响更强。这一发现表明,在优化LLM的多实例处理性能时,应同时考虑上下文长度和实例数量。

英文摘要:

Users often rely on Large Language Models (LLMs) for processing multiple documents or performing analysis over a number of instances. For example, analysing the overall sentiment of a number of movie reviews requires an LLM to process the sentiment of each review individually in order to provide a final aggregated answer. While LLM performance on such individual tasks is generally high, there has been little research on how LLMs perform when dealing with multi-instance inputs. In this paper, we perform a comprehensive evaluation of the multi-instance processing (MIP) ability of LLMs for tasks in which they excel individually. The results show that all LLMs follow a pattern of slight performance degradation for small numbers of instances (approximately 20-100), followed by a performance collapse on larger instance counts. Crucially, our analysis shows that while context length is associated with this degradation, the number of instances has a stronger effect on the final results. This finding suggests that when optimising LLM performance for MIP, attention should be paid to both context length and, in particular, instance count.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑