鲁棒领域自适应的快速批量核范数最大化和最小化
Fast Batch Nuclear-norm Maximization and Minimization for Robust Domain Adaptation
摘要：
由于视觉领域自适应中的领域差异性，当目标域中决策边界附近的数据密度较高时，源模型的性能会下降。一种常见的解决方案是最小化香农熵以将决策边界推离高密度区域。然而，熵最小化也导致预测多样性的严重降低，并且不幸地给领域自适应带来危害。在本文中，我们研究的预测的区别和多样性，通过研究的结构的分类输出矩阵的随机选择的数据批。通过理论分析发现，预测的区分度和多样性可以分别用批量输出矩阵的Frobenius范数和秩来度量。核范数是前者的上界，也是后者的凸近似。相应地，我们提出了批核范数最大化和最小化，它执行核范数最大化的目标输出矩阵，以提高目标的预测能力，和核范数最小化的源批量输出矩阵，以增加源领域知识的适用性。我们进一步用L_{1,2} -范数逼近核范数，并设计了多批优化算法，以获得大类问题的稳定解。快速逼近算法具有O（n^2）的计算复杂性和较好的收敛性。实验结果表明，该方法在三种典型的领域自适应场景下均能提高自适应的准确性和鲁棒性。该代码可在https://github.com/cuishuhao/BNM上获得。
英文摘要：
Due to the domain discrepancy in visual domain adaptation, the performance of source model degrades when bumping into the high data density near decision boundary in target domain. A common solution is to minimize the Shannon Entropy to push the decision boundary away from the high density area. However, entropy minimization also leads to severe reduction of prediction diversity, and unfortunately brings harm to the domain adaptation. In this paper, we investigate the prediction discriminability and diversity by studying the structure of the classification output matrix of a randomly selected data batch. We find by theoretical analysis that the prediction discriminability and diversity could be separately measured by the Frobenius-norm and rank of the batch output matrix. The nuclear-norm is an upperbound of the former, and a convex approximation of the latter. Accordingly, we propose Batch Nuclear-norm Maximization and Minimization, which performs nuclear-norm maximization on the target output matrix to enhance the target prediction ability, and nuclear-norm minimization on the source batch output matrix to increase applicability of the source domain knowledge. We further approximate the nuclear-norm by L_{1,2}-norm, and design multi-batch optimization for stable solution on large number of categories. The fast approximation method achieves O(n^2) computational complexity and better convergence property. Experiments show that our method could boost the adaptation accuracy and robustness under three typical domain adaptation scenarios. The code is available at https://github.com/cuishuhao/BNM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
