领域感知剪枝：通过正则化概率掩码实现稀疏性与领域泛化
Domain-Aware Pruning: Sparsity and Domain Generalization via Regularized Probabilistic Masking
- University of Tehran（德黑兰大学）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
本研究提出算法无关的领域感知剪枝框架，通过正则化概率掩码识别领域不变子网络，在实现高稀疏性的同时，提升分布外鲁棒性与对抗鲁棒性，且模型可解释性强。
中文摘要 AI 辅助
领域泛化（DG）与神经网络剪枝通常被视为不同目标，分别针对分布外（OOD）鲁棒性和模型效率。本研究通过引入领域感知剪枝（Domain-Aware Pruning，DAP）框架弥合二者差距，该框架利用网络稀疏性作为隐式增强对未见领域泛化的机制。与标准二值掩码优化不同，DAP学习连续参数保留概率p∈[0,1]，将网络压缩构建为连续概率掩码问题。通过引入正则化目标，在掩码训练期间主动惩罚领域敏感权重的保留，DAP识别出领域不变子网络。在五个DG基准数据集上的实验结果表明，DAP在实现显著稀疏性的同时，始终匹配或超过其密集模型的OOD性能。至关重要的是，DAP是一种算法无关框架，可无缝集成到现有DG流程中，无需事后微调。除效率与泛化性外，研究还显示DAP天然提升对抗扰动鲁棒性，并产生高度可解释的模型，其中保留的权重可靠封装最具领域不变性和任务关键性的表征。
英文摘要
Domain generalization (DG) and neural network pruning are conventionally treated as distinct objectives, targeting out-of-distribution (OOD) robustness and model efficiency, respectively. In this work, we bridge this gap by introducing Domain-Aware Pruning (DAP), a framework that leverages network sparsity as a mechanism to implicitly enhance generalization to unseen domains. Diverging from standard binary mask optimization, DAP learns a continuous parameter retention probability $p \in [0, 1]$, framing network compression as a continuous probabilistic masking problem. By introducing a regularization objective that actively penalizes the retention of domain-sensitive weights during the mask training, DAP identifies a domain-invariant subnetwork. Empirical results across five DG benchmark datasets demonstrate that DAP achieves significant sparsity while consistently matching or exceeding the OOD performance of its dense counterparts. Crucially, DAP is an algorithm-agnostic framework that integrates seamlessly with existing DG pipelines without necessitating post-hoc fine-tuning. Beyond efficiency and generalization, we show that DAP natively provides increased robustness to adversarial perturbations and yields highly interpretable models, where the retained weights reliably encapsulate the most domain-invariant and task-critical representations.
