无梯度适应：仿射统计传输及其证书的作用

中文摘要 AI 辅助

测试时适应（TTA）通常假设模型参数可在推理时更新，但该假设对于仅推理加速器、冻结模型或第三方模型以及内存受限部署具有局限性，且基于标准批归一化（BatchNorm）的TTA配置在无BatchNorm的架构上可能失效。本文研究学习模型必须保持冻结时的适应问题，提出CASTER这一无梯度方法：它在判别子空间中存储源类别统计量，从目标批次矩估计类别共享仿射变换，并在分类前解析传输源类别分布。CASTER无需反向传播、优化器状态或存储的源特征库。在4种主干网络和7个数据集上，它在28种主干-数据集设置中的27种优于相同冻结特征上的k近邻（k-NN），同时状态量中位数减少18倍。仿射传输并非总是可靠：在ImageNet-C上，批次包含1000类各64个样本时，无条件传输的Top-1准确率下降21.2个百分点。因此，本文提出经验残差-间隔传输性证书：在307个评估单元中，所有下降超过10个百分点的传输其证书值均高于3.9，尽管良性与破坏性场景未完全分离；门控机制将无条件传输的平均-3.35点效应转为+1.69点增益，且在宽阈值范围内性能与最佳阈值相差不超过0.3点。最后，本文表明该证书具有机制特异性：应用于Tent方法时，仅接受4.3%的更新，保留Tent可用增益的0.6%。这些结果确立了CASTER作为冻结模型部署的轻量适应机制，同时明确了其安全信号的有效与无效场景。

英文摘要

Test-time adaptation (TTA) typically assumes that model parameters can be updated at inference time. This assumption is restrictive for inference-only accelerators, frozen or third-party models, and memory-constrained deployments, and standard BatchNorm-based TTA configurations may also become inactive on architectures without BatchNorm. We study adaptation when the learned model must remain frozen. We introduce CASTER, a gradient-free method that stores source class statistics in a discriminative subspace, estimates a class-shared affine transformation from target-batch moments, and analytically transports the source class distributions before classification. CASTER requires no backward pass, optimizer state, or stored source feature bank. Across four backbones and seven datasets, it outperforms k-NN on identical frozen features in 27 of 28 backbone-dataset settings while retaining a median of 18x less state. Affine transport is not always reliable. On ImageNet-C, where batches contain only 64 samples for 1000 classes, unconditional transport loses 21.2 top-1 points. We therefore introduce an empirical residual-to-margin transportability certificate. Across 307 evaluation cells, every transport losing more than 10 points has certificate value above 3.9, although benign and destructive regimes are not perfectly separated. Gating converts an average $-3.35$-point effect of unconditional transport into a +1.69-point gain, and performance remains within 0.3 points of the best threshold over a broad threshold range. Finally, we show that this certificate is mechanism-specific: when applied to Tent, it accepts only $4.3\%$ of updates and preserves 0.6% of Tent's available gain. These results position CASTER as a lightweight adaptation mechanism for frozen-model deployment, together with an explicit account of when its safety signal is informative and when it is not.

发表机构

• Talan Research Center（塔兰研究中心）

机构由 AI 辅助整理，请以论文原文为准。