arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.13552cs.CLcs.AI
收藏

无需训练的测试时对比学习用于大语言模型

Training-Free Test-Time Contrastive Learning for Large Language Models

South China University of Technology(南方科技大学);Pazhou Laboratory(琶洲实验室)

Kaiwen Zheng, Kai Zhou, Jinwu Hu, Te Gu, Mingkai Peng, Fei Liu

AI总结:

本文提出无需训练的测试时对比学习(TF-TTCL),通过动态'探索-反思-引导'循环提升冻结大语言模型的在线推理能力,优于零样本基线和代表性测试时适应方法。

摘要:

大语言模型(LLMs)表现出强大的推理能力,但在分布偏移下性能往往下降。现有的测试时适应(TTA)方法依赖梯度更新,需要白盒访问并带来大量开销,而无需训练的替代方法要么静态要么依赖外部指导。本文提出无需训练的测试时对比学习TF-TTCL,是一种无需训练的适应框架,使冻结的LLM通过自身推理经验进行学习。具体而言,TF-TTCL通过三个核心模块实现动态'探索-反思-引导'循环:1)语义查询增强通过多代理角色扮演多样化问题视角,生成不同的推理轨迹;2)对比经验蒸馏捕捉优劣轨迹之间的语义差距,将其蒸馏为显式文本规则;3)上下文规则检索在推理过程中激活这些存储的规则,动态引导冻结的LLM向稳健的推理模式发展,同时避免已观察到的错误。在封闭式推理任务和开放式评估任务上的广泛实验表明,TF-TTCL在在线评估下一致优于强大的零样本基线和代表性TTA方法。代码可在https://github.com/KevinSCUTer/TF-TTCL获取。

英文摘要:

Large language models (LLMs) demonstrate strong reasoning capabilities, but their performance often degrades under distribution shift. Existing test-time adaptation (TTA) methods rely on gradient-based updates that require white-box access and need substantial overhead, while training-free alternatives are either static or depend on external guidance. In this paper, we propose Training-Free Test-Time Contrastive Learning TF-TTCL, a training-free adaptation framework that enables a frozen LLM to improve online by distilling supervision from its own inference experiences. Specifically, TF-TTCL implements a dynamic "Explore-Reflect-Steer" loop through three core modules: 1) Semantic Query Augmentation first diversifies problem views via multi-agent role-playing to generate different reasoning trajectories; 2) Contrastive Experience Distillation then captures the semantic gap between superior and inferior trajectories, distilling them into explicit textual rules; and 3) Contextual Rule Retrieval finally activates these stored rules during inference to dynamically steer the frozen LLM toward robust reasoning patterns while avoiding observed errors. Extensive experiments on closed-ended reasoning tasks and open-ended evaluation tasks demonstrate that TF-TTCL consistently outperforms strong zero-shot baselines and representative TTA methods under online evaluation. Code is available at https://github.com/KevinSCUTer/TF-TTCL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑