Monte-Carlo planning and Reinforcement Learning (RL) are essential to sequential decision making. The recent AlphaGo and AlphaZero algorithms have shown how to successfully combine these two paradigms in order to solve large scale sequential decision problems. These methodologies exploit a variant of the well-known UCT algorithm to trade off exploitation of good actions and exploration of unvisited states, but their empirical success comes at the cost of poor sample-efficiency and high computation time. In this paper, we overcome these limitations by considering convex regularization in Monte-Carlo Tree Search (MCTS), which has been successfully used in RL to efficiently drive exploration. First, we introduce a unifying theory on the use of generic convex regularizers in MCTS, deriving the regret analysis and providing guarantees of exponential convergence rate. Second, we exploit our theoretical framework to introduce novel regularized backup operators for MCTS, based on the relative entropy of the policy update, and on the Tsallis entropy of the policy. Finally, we empirically evaluate the proposed operators in AlphaGo and AlphaZero on problems of increasing dimensionality and branching factor, from a toy problem to several Atari games, showing their superiority w.r.t. representative baselines.
Non-convex Regularizations for Feature Selection in Ranking With Sparse SVM
Institut de Recherche en Informatique de Toulouse（图卢兹计算机科学研究所）；CNRS（法国国家科学研究中心）；Université de Toulouse（图卢兹大学）；Nomao SA（Nomao公司）；Laboratoire Lagrange（拉格朗日实验室）；Université de Nice-Sophia Antipolis（尼斯-索菲亚·安提波利斯大学）；INSA de Rouen（鲁昂国立应用科学学院）；Institut de Mathématiques de Toulouse（图卢兹数学研究所）；Institut Universitaire de Formation de Maîtres（大学教师培训学院）
Don't relax: early stopping for convex regularization
Ecole Politechnique（巴黎综合理工学院）；Istituto Italiano di Tecnologia（意大利技术研究院）；Massachusetts Institute of Technology（麻省理工学院）；Università di Genova（热那亚大学）；Politecnico di Milano（米兰理工大学）；EPFL（洛桑联邦理工学院）
Non-Convex Optimization with Spectral Radius Regularization
Engineering Sciences and Applied Mathematics, Northwestern University（西北大学工程科学与应用数学系）；Industrial Engineering and Management Sciences, Northwestern University（西北大学工业工程与管理科学系）；Preventive Medicine (Health and Biomedical Informatics), Northwestern University（西北大学预防医学（健康与生物医学信息学））
Convex regularization in statistical inverse learning problems
University of Helsinki（赫尔辛基大学）；University of Cambridge（剑桥大学）；Friedrich-Alexander Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）；Lappeenranta University of Technology（拉彭兰塔理工大学）；University of Genoa（热那亚大学）
Successive Convex Approximation Algorithms for Sparse Signal Estimation with Nonconvex Regularizations
Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg（卢森堡大学安全、可靠性与信任跨学科中心）；Technische Universität Darmstadt（达姆施塔特工业大学）
Weakly Convex Ridge Regularization for 3D Non-Cartesian MRI Reconstruction
Faculty of Mathematics, Chemnitz University of Technology（开姆尼茨工业大学数学学院）；NeuroSpin, Frédéric Joliot Institute for Life Sciences, CEA Paris-Saclay（法国原子能委员会巴黎-萨克雷大学弗雷德里克·约里奥生命科学研究所神经自旋中心）；MIND, Inria Saclay Centre（法国国家信息与自动化研究所萨克雷中心MIND团队）；Siemens Healthineers（西门子医疗）
Regularity as Regularization: Smooth and Strongly Convex Brenier Potentials in Optimal Transport
CREST（法国国家科学研究中心经济与统计研究中心）；ENSAE（法国国家统计与经济管理学院）；Institut Polytechnique de Paris（巴黎综合理工学院）；CNRS（法国国家科学研究中心）；ENS, PSL Research University（巴黎高等师范学院，巴黎文理研究大学）；Google Brain（谷歌大脑）
Hyperspectral Image Denoising Using Non-convex Local Low-rank and Sparse Separation with Spatial-Spectral Total Variation Regularization
Qingdao University（青岛大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Electronic Science and Technology of China（电子科技大学）；University of Kentucky（肯塔基大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
Global Convergence of Stochastic Gradient Hamiltonian Monte Carlo for Non-Convex Stochastic Optimization: Non-Asymptotic Performance Bounds and Momentum-Based Acceleration
The Chinese University of Hong Kong（香港中文大学）；Rutgers University（罗格斯大学）；Florida State University（佛罗里达州立大学）
随机梯度Hamilton Monte Carlo（SGHMC）是一种动量随机梯度算法，在随机梯度中加入受控的、适当缩放的高斯噪声，使迭代过程趋于全局极小值。许多工作报告了其在解决随机非凸优化问题的实践中的经验成功，特别是它已被观察到优于过阻尼Langevin蒙特卡罗方法，如随机梯度Langevin动力学（SGLD）在许多应用中。虽然SGHMC的渐近整体收敛性质是众所周知的，其有限时间性能还没有得到很好的理解。在这项工作中，我们研究了两个变种的SGHMC的基础上两个替代离散的欠阻尼朗之万扩散。我们提供了有限时间性能界的整体收敛的SGHMC解决显式常数随机非凸优化问题的变种。我们的研究结果导致人口和经验风险最小化问题的非渐近保证。对于一个固定的目标精度水平，对一类非凸问题，我们得到的复杂性界SGHMC可以比SGLD更紧。这些结果表明，在全局非凸优化的背景下，动量加速是可能的。
