通过多示例提示进行测试时适应：益处、限制与陷阱
Test-Time Adaptation via Many-Shot Prompting: Benefits, Limits, and Pitfalls
AI总结：
研究通过多示例提示在不同任务和模型中分析测试时适应的性能变化，探讨其有效性及限制，指出其在结构化任务中的优势及在开放生成任务中的不足。
摘要：
测试时适应使大语言模型在推理时无需更新参数即可调整行为。常见方法是多示例提示，通过注入大量上下文学习（ICL）示例作为输入空间的测试时更新。尽管增加示例能提升性能，但该更新机制的可靠性及限制尚不明确，尤其对开源模型。本文通过跨任务和模型分析，探讨性能随更新规模、示例顺序和选择策略的变化。进一步研究动态和强化ICL作为替代测试时更新策略，控制注入信息及约束模型行为。发现多示例提示在信息增益高的结构化任务中有效，但对选择策略敏感，且在开放生成任务中收益有限。总体上，本文刻画了基于提示的测试时适应的实用限制，并概述了输入空间更新的有益与有害场景。
英文摘要：
Test-time adaptation enables large language models (LLMs) to modify their behavior at inference without updating model parameters. A common approach is many-shot prompting, where large numbers of in-context learning (ICL) examples are injected as an input-space test-time update. Although performance can improve as more demonstrations are added, the reliability and limits of this update mechanism remain poorly understood, particularly for open-source models. We present an empirical study of many-shot prompting across tasks and model backbones, analyzing how performance varies with update magnitude, example ordering, and selection policy. We further study Dynamic and Reinforced ICL as alternative test-time update strategies that control which information is injected and how it constrains model behavior. We find that many-shot prompting is effective for structured tasks where demonstrations provide high information gain, but is highly sensitive to selection strategy and often shows limited benefits for open-ended generation tasks. Overall, we characterize the practical limits of prompt-based test-time adaptation and outline when input-space updates are beneficial versus harmful.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
