King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);The University of Hong Kong(香港大学);University of Edinburgh(爱丁堡大学);Moscow Institute of Physics and Technology(莫斯科物理技术学院)
We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models. One of the most celebrated methods in this context is the SAGA algorithm. Despite years of research on the topic, a general-purpose version of SAGA---one that would include arbitrary importance sampling and minibatching schemes---does not exist. We remedy this situation and propose a general and flexible variant of SAGA following the {\em arbitrary sampling} paradigm. We perform an iteration complexity analysis of the method, largely possible due to the construction of new stochastic Lyapunov functions. We establish linear convergence rates in the smooth and strongly convex regime, and under a quadratic functional growth condition (i.e., in a regime not assuming strong convexity). Our rates match those of the primal-dual method Quartz for which an arbitrary sampling analysis is available, which makes a significant step towards closing the gap in our understanding of complexity of primal and dual methods for finite sum problems.
A Short and Unified Convergence Analysis of the SAG, SAGA, and IAG Algorithms
Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, USA(北卡罗来纳州立大学电气与计算机工程系);Department of Electrical and Computer Engineering, University of California, San Diego, USA(加州大学圣地亚哥分校电气与计算机工程系)
Stochastic Primal Dual Coordinate Method with Non-Uniform Sampling Based on Optimality Violations
Nagoya Institute of Technology(名古屋工业大学);RIKEN Center for Advanced Intelligence Project(理化学研究所革新智能综合研究中心);National Institute for Materials Science(国立材料科学研究所)
Discrete Equilibrium Sampling with Arbitrary Nonequilibrium Processes
D-Wave Systems Inc.(D-Wave系统公司)
我们提出了一个新的框架进行统计抽样,期望估计,分区函数近似使用\emph{arbitrary}启发式随机过程定义在离散状态空间。使用一个高度并行的结构,我们称之为\emph{sequential constraining process},我们能够同时产生的启发式过程的状态,并准确地估计其概率,即使它们太小,实际上是通过直接计数推断。在表明理论上正确的重要性抽样和马尔可夫链蒙特卡罗是可能的顺序约束过程中,我们将其集成到一个方法称为\emph{state space sampling},从计算机科学的抽样范围内的状态空间搜索的想法。该方法包括动态数据结构,该动态数据结构构造由受精度约束的启发式过程生成的统计的鲁棒贝叶斯模型,该精度约束为后验库尔贝克-莱布勒散度。从动态结构中采样通常会产生部分状态,这是通过递归调用启发式算法来细化结构并恢复采样来完成的。我们的各种伊辛模型的实验表明,状态空间采样,使启发式状态生成与准确的概率估计,证明了模拟退火过程的收敛到玻尔兹曼分布的运行长度增加。因此,迄今为止,前所未有的直接重要性抽样使用一般随机过程的\emph{final}(边际)分布是允许的,潜在地增加了蒙特卡罗从业者的处置的算法的范围。
Learning with the Weighted Trace-norm under Arbitrary Sampling Distributions
University of Chicago(芝加哥大学);Massachusetts Institute of Technology(麻省理工学院);Microsoft Research New England(微软新英格兰研究院);Toyota Technological Institute at Chicago(芝加哥丰田技术研究院)
FedGS: Federated Graph-based Sampling with Arbitrary Client Availability
Fujian Key Laboratory of Sensing and Computing for Smart Cities, School of Informatics, Xiamen University(厦门大学信息学院福建省智慧城市感知与计算重点实验室);School of Computing and Information Systems, University of Melbourne(墨尔本大学计算与信息系统学院)
Christoffel-DPS: Optimal sensor placement in diffusion posterior sampling for arbitrary distributions
Department of Applied Mathematics and Theoretical Physics(应用数学与理论物理系);University of Cambridge(剑桥大学);Department of Mathematics(数学系);Simon Fraser University(西蒙弗雷泽大学)
Normalizing flows for lattice gauge theory in arbitrary space-time dimension
Massachusetts Institute of Technology(麻省理工学院);New York University(纽约大学);DeepMind(深度思维);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);University of Bern(伯尔尼大学)
Satellite Observations Guided Diffusion Model for Accurate Meteorological States at Arbitrary Resolution
Fudan University(复旦大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Beihang University(北京航空航天大学);Columbia University(哥伦比亚大学);Nanjing University of Information Science and Technology(南京信息工程大学)