arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.04397cs.LGstat.ML
收藏

李群贝叶斯学习规则

The Lie-Group Bayesian Learning Rule

RIKEN Center for AI Project(理化学研究所人工智能项目中心)

Eren Mehmet Kıral, Thomas Möllenhoff, Mohammad Emtiyaz Khan

摘要:

贝叶斯学习规则为通用算法设计提供了一个框架,但由于以下三个原因可能难以使用。首先,它需要指数族的特定参数化。其次,它使用了难以计算的梯度。第三,它的更新可能并不总是停留在流形上。我们解决这些困难,提出了一个扩展的基础上,李群后验参数化通过变换的任意基础分布和更新通过该组的指数映射。这简化了许多情况下的所有三个困难,通过群的作用提供灵活的参数化,通过重新参数化提供简单的梯度计算,以及始终停留在流形上的更新。我们使用新的学习规则推导出一种新的深度学习算法,该算法具有理想的生物学上合理的属性来学习稀疏特征。我们的工作开辟了一个新的前沿设计的新算法,利用李群结构。

英文摘要:

The Bayesian Learning Rule provides a framework for generic algorithm design but can be difficult to use for three reasons. First, it requires a specific parameterization of exponential family. Second, it uses gradients which can be difficult to compute. Third, its update may not always stay on the manifold. We address these difficulties by proposing an extension based on Lie-groups where posteriors are parametrized through transformations of an arbitrary base distribution and updated via the group's exponential map. This simplifies all three difficulties for many cases, providing flexible parametrizations through group's action, simple gradient computation through reparameterization, and updates that always stay on the manifold. We use the new learning rule to derive a new algorithm for deep learning with desirable biologically-plausible attributes to learn sparse features. Our work opens a new frontier for the design of new algorithms by exploiting Lie-group structures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑