arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.12017stat.MLcs.LGstat.CO
收藏

通过拼接迭代进行稀疏性约束优化

Sparsity-Constraint Optimization via Splicing Iteration

School of Mathematics, University of Birmingham, Birmingham, UK(英国伯明翰大学数学学院);Saw Swee Hock School of Public Health, National University of Singapore, Singapore(新加坡国立大学 Saw Swee Hock 公共卫生学院);Department of Statistics and Finance/International Institute of Finance, School of Management, University of Science and Technology of China, Hefei, Anhui, China(中国科学技术大学管理学院统计与金融系/国际金融研究院);School of Statistics and Information, Shanghai University of International Business and Economics, Shanghai, China(上海国际经贸管理大学统计与信息学院)

Jin Zhu, Junxian Zhu, Zezhi Wang, Borui Tang, Hongmei Lin, Xueqin Wang

AI总结:

本文提出SCOPE算法,通过拼接操作优化强凸光滑函数,无需调参,理论证明线性收敛并恢复真实支持集,应用于稀疏二次优化、稀疏分类器学习和稀疏马尔可夫网络恢复。

摘要:

稀疏性约束优化在信号处理、统计和机器学习中普遍存在。现有硬阈值算法依赖连续步长参数保证收敛。本文提出SCOPE(通过拼接迭代进行稀疏性约束优化)算法,能够优化在低维子空间上强凸光滑的非线性可微目标函数。SCOPE将梯度步替代为由目标值直接引导的拼接操作,从而消除对任何连续超参数的调优需求。理论上,它实现了线性收敛率并在稀疏水平正确指定时恢复真实支持集。我们还建立了无需依赖限制等距性质条件的并行理论结果。我们应用SCOPE的通用性和强大性解决稀疏二次优化、学习稀疏分类器和恢复二元变量的稀疏马尔可夫网络。通过我们的C++实现,这些任务的数值实验表明,SCOPE在支持恢复性能上表现优异,证实了其算法效率和理论保证。

英文摘要:

Sparsity-constrained optimization underlies many problems in signal processing, statistics, and machine learning. State-of-the-art hard-thresholding (HT) algorithms rely on an appropriately selected continuous step-size parameter to ensure convergence. In this paper, we propose a naturally convergent iterative algorithm, SCOPE (Sparsity-Constrained Optimization via sPlicing itEration). The algorithm is capable of optimizing nonlinear differentiable objective functions that are strongly convex and smooth on low-dimensional subspaces. SCOPE replaces the gradient step with a splicing operation guided directly by the objective value, thereby eliminating the need to tune any continuous hyperparameter. Theoretically, it achieves a linear convergence rate and recovers the true support set when the sparsity level is correctly specified. We also establish parallel theoretical results without relying on restricted-isometry-property-type conditions. We apply SCOPE's versatility and power to solve sparse quadratic optimization, learn sparse classifiers, and recover sparse Markov networks for binary variables. With our C++ implementation of SCOPE, numerical experiments on these tasks show that it achieves superior support recovery performance, confirming both its algorithmic efficiency and theoretical guarantees.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑