FOZO：面向测试时间适应的前向零阶提示优化
FOZO: Forward-Only Zeroth-Order Prompt Optimization for Test-Time Adaptation
AI总结：
本文提出FOZO，一种无需反向传播的测试时间适应方法，通过优化中间特征统计和预测熵，实现高效稳定适应，实验显示其在ImageNet-C上达到59.52%的Top-1准确率，优于主流方法。
摘要：
测试时间适应（TTA）对于使深度学习模型处理现实数据分布偏移至关重要。然而，当前方法存在显著局限：基于反向传播的方法不适合低端部署设备，因其计算和内存需求高且会修改模型权重；而传统无反向传播方法适应能力受限。本文提出前向仅零阶优化（FOZO），一种新颖且实用的无反向传播TTA范式。FOZO利用内存高效的零阶提示优化，由优化中间特征统计和预测熵的目标驱动。为确保在分布外数据流上的高效稳定适应，我们引入动态衰减的扰动尺度进行零阶梯度估计，并在TTA数据流假设下理论证明其收敛性。在ImageNet-C、ImageNet-R和ImageNet-Sketch上的持续适应实验表明，FOZO的性能优越，达到ImageNet-C（5K，级别5）59.52%的Top-1准确率，优于主流梯度方法和最先进的前向仅FOA（58.13%）。此外，FOZO在量化（INT8）模型上表现出强泛化能力。这些发现表明，FOZO是资源受限场景下TTA部署的有力解决方案。
英文摘要：
Test-Time Adaptation (TTA) is essential for enabling deep learning models to handle real-world data distribution shifts. However, current approaches face significant limitations: backpropagation-based methods are not suitable for low-end deployment devices, due to their high computation and memory requirements, as well as their tendency to modify model weights during adaptation; while traditional backpropagation-free techniques exhibit constrained adaptation capabilities. In this work, we propose Forward-Only Zeroth-Order Optimization (FOZO), a novel and practical backpropagation-free paradigm for TTA. FOZO leverages a memory-efficient zeroth-order prompt optimization, which is led by objectives optimizing both intermediate feature statistics and prediction entropy. To ensure efficient and stable adaptation over the out-of-distribution data stream, we introduce a dynamically decaying perturbation scale during zeroth-order gradient estimation and theoretically prove its convergence under the TTA data stream assumption. Extensive continual adaptation experiments on ImageNet-C, ImageNet-R, and ImageNet-Sketch demonstrate FOZO's superior performance, achieving 59.52% Top-1 accuracy on ImageNet-C (5K, level 5) and outperforming main gradient-based methods and SOTA forward-only FOA (58.13%). Furthermore, FOZO exhibits strong generalization on quantized (INT8) models. These findings demonstrate that FOZO is a highly competitive solution for TTA deployment in resource-limited scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
