面向标签偏移鲁棒测试时自适应的通道选择性归一化
Channel-Selective Normalization for Label-Shift Robust Test-Time Adaptation
AI总结：
针对测试时自适应在标签分布偏移下易失效的问题，提出通道选择性归一化方法，通过仅自适应对标签偏移不敏感的通道来降低风险，在多个分类任务中验证了其有效性和鲁棒性。
摘要：
深度神经网络在许多不同任务中都有有用的应用，然而其性能可能会受到数据分布变化的严重影响。例如，在生物医学领域，其性能可能会受到训练和测试数据集之间数据变化（不同机器、人群）的影响。为了确保对真实世界场景的鲁棒性和泛化能力，测试时自适应最近被研究作为一种在推理期间调整模型以适应新数据分布的方法。测试时批归一化是一种简单且流行的方法，在领域偏移基准上取得了令人信服的性能。它通过在测试批次上重新计算批归一化统计量来实现。先前的工作集中在分析与训练数据具有相同标签分布的测试数据上。然而，在许多实际应用中，该技术容易受到标签分布偏移的影响，有时会产生灾难性的失败。这在部署中应用测试时自适应方法带来了风险。我们提出通过仅选择性地自适应深度网络中的通道来应对这一挑战，以最小化对标签偏移敏感的剧烈自适应。我们的选择方案基于我们通过实验得出的两个原则：（1）网络的较后层对标签偏移更敏感（2）单个特征可能对特定类别敏感。我们将所提出的技术应用于三个分类任务，包括CIFAR10-C、Imagenet-C和脂肪肝诊断，在这些任务中我们探索了协变量和标签分布偏移。我们发现我们的方法能够带来TTA的益处，同时显著降低了其他方法中常见的失败风险，并且对超参数的选择具有鲁棒性。
英文摘要：
Deep neural networks have useful applications in many different tasks, however their performance can be severely affected by changes in the data distribution. For example, in the biomedical field, their performance can be affected by changes in the data (different machines, populations) between training and test datasets. To ensure robustness and generalization to real-world scenarios, test-time adaptation has been recently studied as an approach to adjust models to a new data distribution during inference. Test-time batch normalization is a simple and popular method that achieved compelling performance on domain shift benchmarks. It is implemented by recalculating batch normalization statistics on test batches. Prior work has focused on analysis with test data that has the same label distribution as the training data. However, in many practical applications this technique is vulnerable to label distribution shifts, sometimes producing catastrophic failure. This presents a risk in applying test time adaptation methods in deployment. We propose to tackle this challenge by only selectively adapting channels in a deep network, minimizing drastic adaptation that is sensitive to label shifts. Our selection scheme is based on two principles that we empirically motivate: (1) later layers of networks are more sensitive to label shift (2) individual features can be sensitive to specific classes. We apply the proposed technique to three classification tasks, including CIFAR10-C, Imagenet-C, and diagnosis of fatty liver, where we explore both covariate and label distribution shifts. We find that our method allows to bring the benefits of TTA while significantly reducing the risk of failure common in other methods, while being robust to choice in hyperparameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
