基于检索的上下文学习：一种领域自适应框架

中文摘要 AI 辅助

上下文检索（ICR）是一种基于检索的上下文学习（ICL）形式，其中示范样本根据与查询的相似性从源数据库中检索，而非独立采样。在本工作中，我们将ICR表述为一类领域自适应问题，其中数据库的源分布$P$可能不同于测试查询-标签对的目标分布$Q$。我们在一类灵活分布偏移下研究ICR的性能，该分布偏移显著扩展了先前工作，并建立了量化该学习范式优势与陷阱的理论保证。我们的理论通过合成任务和语言任务的实验得到验证。

英文摘要

In-context retrieval (ICR) is a retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based on similarity to the query, rather than sampled independently. In this work, we formulate ICR as a type of domain adaptation problem, where the source distribution $P$ of the database may differ from the target distribution $Q$ of the test query-label pair. We investigate the performance of ICR under a flexible class of distributional shifts that substantially extends prior work \citep{li2024fine,guo2025retrieval}, and establish theoretical guarantees that quantify the benefits and pitfalls of this learning paradigm. Our theory is verified by experiments on synthetic and language tasks.

发表机构

• University of Michigan（密歇根大学）
• New Jersey Institute of Technology（新泽西理工学院）
• Northwestern University（西北大学）

机构由 AI 辅助整理，请以论文原文为准。