卷积ResNets能近似保持输入距离吗？频率分析视角
Can convolutional ResNets approximately preserve input distances? A frequency analysis perspective
摘要：
约束为双Lipschitz的ResNet，即近似距离保持，是最近提出的神经模型中确定性不确定性量化技术的关键组成部分。我们发现，最近的正则化计划试图执行这样的约束的理论理由遭受一个关键的缺陷-正则化计划和双Lipschitzness之间的理论联系是唯一有效的条件下，不持有在实践中，使现有的理论使用有限，尽管这些模型的强大的经验表现。我们提供了一个理论上的解释，这些正规化计划的有效性，使用频率分析的角度来看，在温和的条件下，这些计划将强制执行较低的Lipschitz约束的低频投影的图像。然后，我们提供了支持我们的理论主张的经验证据，并进行进一步的实验，这些实验表明，当我们的证明中的一些数学假设被放松时，我们更广泛的结论似乎成立，对应于先前工作中使用的设置。此外，我们提出了一个简单的构造性算法来搜索距离保持条件的反例，并讨论了我们的理论对未来模型设计的可能影响。
英文摘要：
ResNets constrained to be bi-Lipschitz, that is, approximately distance preserving, have been a crucial component of recently proposed techniques for deterministic uncertainty quantification in neural models. We show that theoretical justifications for recent regularisation schemes trying to enforce such a constraint suffer from a crucial flaw -- the theoretical link between the regularisation scheme used and bi-Lipschitzness is only valid under conditions which do not hold in practice, rendering existing theory of limited use, despite the strong empirical performance of these models. We provide a theoretical explanation for the effectiveness of these regularisation schemes using a frequency analysis perspective, showing that under mild conditions these schemes will enforce a lower Lipschitz bound on the low-frequency projection of images. We then provide empirical evidence supporting our theoretical claims, and perform further experiments which demonstrate that our broader conclusions appear to hold when some of the mathematical assumptions of our proof are relaxed, corresponding to the setup used in prior work. In addition, we present a simple constructive algorithm to search for counter examples to the distance preservation condition, and discuss possible implications of our theory for future model design.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
