arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.02124cs.DScs.NAmath.NA
收藏

高效连续优化的位复杂度

The Bit Complexity of Efficient Continuous Optimization

Mehrdad Ghadiri, Richard Peng, Santosh S. Vempala

AI总结:

本文研究了线性回归、p-范数回归和线性规划等连续优化问题的位复杂度,证明了逆维护子程序的后向稳定性,并利用有界误差预条件子,给出了依赖条件数的多对数因子下的高效近似求解算法。

摘要:

我们分析了基本优化问题(如 linear regression、$p$-norm regression 和 linear programming (LP))高效算法的位复杂度。现有最先进算法均为迭代算法,在算术运算次数方面,它们与当前两个 $n \ imes n$ 矩阵相乘的时间复杂度相匹配(至多相差 polylogarithmic factors)。然而,先前工作通常假设无限精度算术,且由于复杂的 inverse maintenance 技术,这些算法的实际运行时间未知。为解决这些算法的运行时间和位复杂度问题,我们证明了一个核心公共子程序(即 inverse maintenance)是 backward-stable 的。此外,我们展示了求解受约束加权回归问题的迭代方法可以使用 bounded-error pre-conditioners 完成。具体而言,我们证明 LP 可以在矩阵乘法时间乘以取决于矩阵条件数 $κ$ 以及 LP 问题的内外半径的 polylog factors 下被近似求解。$p$-norm regression 可以在矩阵乘法时间乘以 $κ$ 的 polylog factors 下被近似求解。最后,linear regression 可以在输入稀疏性时间乘以 $κ$ 的 polylog factors 下被近似求解。此外,我们通过利用稀疏线性系统的更快速求解器,给出了实现低于矩阵乘法时间求解 $p$-norm regression 的结果。

英文摘要:

We analyze the bit complexity of efficient algorithms for fundamental optimization problems, such as linear regression, $p$-norm regression, and linear programming (LP). State-of-the-art algorithms are iterative, and in terms of the number of arithmetic operations, they match the current time complexity of multiplying two $n$-by-$n$ matrices (up to polylogarithmic factors). However, previous work has typically assumed infinite precision arithmetic, and due to complicated inverse maintenance techniques, the actual running times of these algorithms are unknown. To settle the running time and bit complexity of these algorithms, we demonstrate that a core common subroutine, known as \emph{inverse maintenance}, is backward-stable. Additionally, we show that iterative approaches for solving constrained weighted regression problems can be accomplished with bounded-error pre-conditioners. Specifically, we prove that linear programs can be solved approximately in matrix multiplication time multiplied by polylog factors that depend on the condition number $κ$ of the matrix and the inner and outer radius of the LP problem. $p$-norm regression can be solved approximately in matrix multiplication time multiplied by polylog factors in $κ$. Lastly, linear regression can be solved approximately in input-sparsity time multiplied by polylog factors in $κ$. Furthermore, we present results for achieving lower than matrix multiplication time for $p$-norm regression by utilizing faster solvers for sparse linear systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑