arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.00790cs.LGcs.CV
收藏

关于坐标MLP的正则化

On Regularizing Coordinate-MLPs

University of Adelaide(阿德莱德大学)

Sameera Ramasinghe, Lachlan MacDonald, Simon Lucey

摘要:

我们证明了深度神经网络(用于回归)的典型隐式正则化假设不适用于坐标MLP,这是一种MLP家族,现在在计算机视觉中无处不在,用于表示高频信号。缺乏这种隐式偏置会破坏训练样本之间的平滑插值,并阻碍在具有不同光谱的信号区域之间进行推广。我们通过傅立叶透镜研究了这种行为,发现随着坐标MLP的带宽的增强,除非明确提供合适的先验,否则较低的频率往往会被抑制。基于这些见解,我们提出了一种简单的正则化技术,可以缓解上述问题,可以将其纳入现有网络,而无需任何架构修改。

英文摘要:

We show that typical implicit regularization assumptions for deep neural networks (for regression) do not hold for coordinate-MLPs, a family of MLPs that are now ubiquitous in computer vision for representing high-frequency signals. Lack of such implicit bias disrupts smooth interpolations between training samples, and hampers generalizing across signal regions with different spectra. We investigate this behavior through a Fourier lens and uncover that as the bandwidth of a coordinate-MLP is enhanced, lower frequencies tend to get suppressed unless a suitable prior is provided explicitly. Based on these insights, we propose a simple regularization technique that can mitigate the above problem, which can be incorporated into existing networks without any architectural modifications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑