arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.16852cs.LGmath.MGmath.OCmath.STstat.MLstat.TH
收藏

基于评论器的正则化学习的星几何学

The Star Geometry of Critic-Based Regularizer Learning

University of California, Los Angeles(加州大学洛杉矶分校);Johns Hopkins University(约翰斯·霍普金斯大学);National University of Singapore(新加坡国立大学)

Oscar Leong, Eliza O'Reilly, Yong Sheng Soh

AI总结:

本文研究基于评论器的正则化学习,通过星几何学和对偶 Brunn-Minkowski 理论,将损失解释为对偶混合体积,导出最优正则化器表达式,并分析神经网络架构与星形体规范函数的关系。

摘要:

变分正则化是解决统计推断任务和逆问题的一种经典技术,现代数据驱动方法通过深度神经网络参数化正则化器,展现出令人印象深刻的经验性能。最近的研究沿此方向学习依赖于任务的正则化器。这是通过将测量值和真实数据的信息整合到无监督的、基于评论器的损失函数中来实现的,其中正则化器将低值赋予可能的数据,将高值赋予不太可能的数据。然而,关于通过此过程学习的正则化器的结构及其与两个数据分布的关系,目前几乎没有理论。为了在这一挑战上取得进展,我们启动了一项研究,即优化基于评论器的损失函数,以在特定的正则化器家族上学习正则化器:星形体的规范函数(或 Minkowski 泛函)。该家族包含实践中常用的正则化器,并与深度神经网络参数化的正则化器共享属性。我们具体研究了从概率测度之间统计距离的变分表示导出的基于评论器的损失。通过利用星几何学和对偶 Brunn-Minkowski 理论的工具,我们阐明了这些损失如何被解释为依赖于数据分布的对偶混合体积。这使我们能够在某些情况下导出最优正则化器的精确表达式。最后,我们识别了哪些神经网络架构会产生此类星形体规范函数,以及这些正则化器何时具有有利于优化的属性。更广泛地说,这项工作强调了星几何学的工具如何有助于理解无监督正则化学习的几何结构。

英文摘要:

Variational regularization is a classical technique to solve statistical inference tasks and inverse problems, with modern data-driven approaches parameterizing regularizers via deep neural networks showcasing impressive empirical performance. Recent works along these lines learn task-dependent regularizers. This is done by integrating information about the measurements and ground-truth data in an unsupervised, critic-based loss function, where the regularizer attributes low values to likely data and high values to unlikely data. However, there is little theory about the structure of regularizers learned via this process and how it relates to the two data distributions. To make progress on this challenge, we initiate a study of optimizing critic-based loss functions to learn regularizers over a particular family of regularizers: gauges (or Minkowski functionals) of star-shaped bodies. This family contains regularizers that are commonly employed in practice and shares properties with regularizers parameterized by deep neural networks. We specifically investigate critic-based losses derived from variational representations of statistical distances between probability measures. By leveraging tools from star geometry and dual Brunn-Minkowski theory, we illustrate how these losses can be interpreted as dual mixed volumes that depend on the data distribution. This allows us to derive exact expressions for the optimal regularizer in certain cases. Finally, we identify which neural network architectures give rise to such star body gauges and when do such regularizers have favorable properties for optimization. More broadly, this work highlights how the tools of star geometry can aid in understanding the geometry of unsupervised regularizer learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑