Bregman proximal point algorithm (BPPA) has witnessed emerging machine learning applications, yet its theoretical understanding has been largely unexplored. We study the computational properties of BPPA through learning linear classifiers with separable data, and demonstrate provable algorithmic regularization of BPPA. For any BPPA instantiated with a fixed Bregman divergence, we provide a lower bound of the margin obtained by BPPA with respect to an arbitrarily chosen norm. The obtained margin lower bound differs from the maximal margin by a multiplicative factor, which inversely depends on the condition number of the distance-generating function measured in the dual norm. We show that the dependence on the condition number is tight, thus demonstrating the importance of divergence in affecting the quality of the learned classifiers. We then extend our findings to mirror descent, for which we establish similar connections between the margin and Bregman divergence, together with a non-asymptotic analysis. Numerical experiments on both synthetic and real-world datasets are provided to support our theoretical findings. To the best of our knowledge, the aforementioned findings appear to be new in the literature of algorithmic regularization.
Adaptive Bregman Proximal Stochastic Gradient with a Stabilized Barzilai--Borwein Step Size
University of Utah（犹他大学）；Beijing Normal-Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Guangzhou Nanfang College（广州南方学院）；The Chinese University of Hong Kong（香港中文大学）
The rate of convergence of Bregman proximal methods: Local geometry vs. regularity vs. sharpness
Univ. Grenoble Alpes（格勒诺布尔大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Grenoble INP（格勒诺布尔理工学院）；Institut de Mathématiques de Toulouse（图卢兹数学研究所）；Université de Toulouse（图卢兹大学）；UPS（图卢兹第三大学）；LIG（格勒诺布尔信息与信号处理实验室）
Spurious Stationarity and Hardness Results for Bregman Proximal-Type Algorithms
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong（香港中文大学系统工程与工程管理系）；Sauder School of Business, University of British Columbia（不列颠哥伦比亚大学萨德勒商学院）
Convex-Concave Backtracking for Inertial Bregman Proximal Gradient Algorithms in Non-Convex Optimization
Faculty of Mathematics and Computer Science, Saarland University（萨尔兰大学数学与计算机科学学院）；Institute of Computer Graphics and Vision, Graz University of Technology（格拉茨技术大学计算机图形与视觉研究所）；Faculty of Industrial Engineering, The Technion（技术学院工业工程学院）
New York University（纽约大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；Tandon School of Engineering（坦登工程学院）；Department of Electrical and Computer Engineering（电子与计算机工程系）；Department of Computer Science and Engineering（计算机科学与工程系）；Department of Electrical and Systems Engineering（电气与系统工程系）
An Accelerated Alternating Partial Bregman Algorithm for ReLU-based Matrix Decomposition
School of Mathematics and Computational Science, Xiangtan University（湘潭大学数学与计算科学学院）；School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院）
Second-Order KKT Guarantees for Bregman ADMM in Nonconvex and Non-Lipschitz Optimization
Department of Electrical and Computer Engineering, Iowa State University（电气与计算机工程系，爱荷华州立大学）；Department of Computer Science and Engineering, Ohio State University（计算机科学与工程系，俄亥俄州立大学）；DAMO Academy, Alibaba Group US（阿里巴巴集团美国DAMO研究院）
Fast and Provably Convergent Algorithms for Gromov-Wasserstein in Graph Data
Stanford University（斯坦福大学）；Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong（香港中文大学）；Shanghai University of Finance and Economics（上海财经大学）
Analyzing and Improving Diffusion Models for Time-Series Data Imputation: A Proximal Recursion Perspective
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University（人工智能国家重点实验室、智能科学与技术学院、北京大学）；Zhejiang University（浙江大学）；Beijing Academy of Artificial Intelligence (BAAI)（北京人工智能研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Center for Data Science, Peking University（北京大学数据科学中心）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Pazhou Laboratory (Huangpu), Guangzhou, China（黄埔实验室（华南））
On Universality of Non-Separable Approximate Message Passing Algorithms
Department of Statistics and Data Science, Yale University（统计与数据科学系，耶鲁大学）；Halicioglu Data Science Institute, University of California, San Diego（Halicioglu数据科学研究所，加州大学圣地亚哥分校）
