协方差矩阵上的参数化度量优化

中文摘要 AI 辅助

黎曼度量的选择会强烈影响基于梯度的协方差矩阵优化的收敛性。欧几里得度量、Bures-Wasserstein度量和仿射不变度量是常见的选择，但它们的相对有效性取决于目标函数。我们引入了一个由 $X^{p}LX^{q}+X^{q}LX^{p}=U$ 定义的两参数族，其中 $L$ 在每个切向量 $U$ 处求解，该族在 $(0,0)$、$(1,0)$ 和 $(1,1)$ 处精确包含上述三种度量，并进一步扩展。我们将族成员的选择视为针对给定问题的一种特定预处理方式。为此，我们分析了在解处的黎曼Hessian的条件数。我们证明它满足一个仅通过指数 $r=p+q$ 依赖于 $(p,q)$ 的下界。当欧几里得Hessian是纯幂形式且不混合任何特征方向时，成员 $p=q=r/2$ 达到该下界，并且一个闭式准则识别了其他达到该下界的成员。我们讨论了针对给定问题调整 $r$ 的方法。在真实协方差数据上的实验证实了预测的条件数以及调整 $r$ 的益处。一个任务协方差示例显示了进一步调整形状带来的额外收益。

英文摘要

The choice of Riemannian metric can strongly influence the convergence of gradient-based optimization over covariance matrices. Euclidean, Bures-Wasserstein and affine-invariant metrics are common choices, but their relative effectiveness depends on the objective. We introduce a two-parameter family defined by $X^{p}LX^{q}+X^{q}LX^{p}=U$, solved for $L$ at each tangent vector $U$, that contains all three as exact members, at $(0,0)$, $(1,0)$ and $(1,1)$, and extends past them. We treat the choice of member as a particular way of preconditioning for a given problem. To this end, we analyze the conditioning of the Riemannian Hessian at the solution. We show that it obeys a lower bound that depends on $(p,q)$ only through the exponent $r=p+q$. When the Euclidean Hessian is a pure power that mixes no eigendirections, the member $p=q=r/2$ attains that bound, and a closed-form criterion identifies the other members that do. We discuss ways to tune $r$ for a given problem. Experiments on real covariance data confirm the predicted conditioning and the benefit of tuning $r$. A task covariance example shows a further gain from tuning the shape.

发表机构

• Nanyang Technological University（南洋理工大学）
• Microsoft India（微软印度）
• Indian Institute of Technology Bombay（印度理工学院孟买分校）

机构由 AI 辅助整理，请以论文原文为准。