发表机构
Graz University of Technology; Samsung AI Center-Cambridge; University of Cambridge; Complexity Science Hub（格拉茨工业大学; 三星剑桥人工智能中心; 剑桥大学; 复杂系统科学中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究分析了结构化模型压缩与测试时适应（TTA）的相互作用，发现压缩会导致TTA性能显著下降，其核心原因是表征多样性降低和结构约束，需设计保留适应性的压缩策略。
AI 中文摘要
部署在实际场景中的深度神经网络必须兼具高效性与适应性，这要求同时进行模型压缩与测试时适应（Test-Time Adaptation，TTA）。尽管两者各自已被充分研究，但它们之间的相互作用仍鲜为人知。我们系统分析了结构化压缩如何影响模型在分布偏移下的适应能力，在CIFAR-10-C和ImageNet-C数据集上使用ResNet-18和ViT-Base模型，评估了多种压缩方法与标准TTA技术的组合效果。我们引入了一个诊断框架，用于检验表征表达性与适应子空间兼容性。研究结果揭示了一个一致的差距：尽管压缩模型在监督适应下仍保持较高准确率，但其TTA性能会随压缩程度提升而显著下降。我们证明这一现象源于表征多样性降低和结构约束限制了可恢复性，且这些影响强烈依赖于压缩方法，凸显了设计能保留适应性的压缩策略的必要性。
英文摘要
Deep neural networks deployed in the wild must be both efficient and adaptable, requiring model compression and test-time adaptation (TTA). While both are well studied in isolation, their interaction remains poorly understood. We systematically analyze how structured compression affects a model's ability to adapt under distribution shift. Using ResNet-18 and ViT-Base on CIFAR-10-C and ImageNet-C, we evaluate multiple compression methods combined with standard TTA techniques. We introduce a diagnostic framework that examines representational expressivity and adaptation subspace compatibility. Our results reveal a consistent gap: although compressed models retain high accuracy under supervised adaptation, their TTA performance degrades significantly with increasing compression. We show that this stems from reduced representational diversity and structural constraints that limit recoverability. These effects strongly depend on the compression method, highlighting the need to design compression strategies that preserve adaptability.
