arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.12865cs.LG
收藏

图表示学习的多保真度稳定性

Multi-fidelity Stability for Graph Representation Learning

New York University(纽约大学)

Yihan He, Joan Bruna

摘要:

在基于图表示学习的结构化预测(简称GRL)问题中,算法返回的假设将目标顶点的\emph{receptive field}中的特征集映射到其标签上。为了理解这些算法的可学习性,我们引入了一种较弱形式的一致稳定性称为\emph{multi-fidelity stability},并给出了弱依赖图的学习保证。我们证明了当感受野稀疏时,~ \citet{london2016stability}关于单样本推广的主张对GRL成立。此外,我们还研究了两个流行的算法:\textbf{(1)}随机梯度下降在凸和非凸景观下的稳定性诱导界。在这个例子中,我们提供了高度依赖于由算法构造的感受野的稀疏性的非渐近界。\textbf{(2)} 1层线性等变GNN上的约束回归问题。在这个例子中,我们提出了两种类型的稳定性之间的差异,证明了多保真度设计的下限。

英文摘要:

In the problem of structured prediction with graph representation learning (GRL for short), the hypothesis returned by the algorithm maps the set of features in the \emph{receptive field} of the targeted vertex to its label. To understand the learnability of those algorithms, we introduce a weaker form of uniform stability termed \emph{multi-fidelity stability} and give learning guarantees for weakly dependent graphs. We testify that ~\citet{london2016stability}'s claim on the generalization of a single sample holds for GRL when the receptive field is sparse. In addition, we study the stability induced bound for two popular algorithms: \textbf{(1)} Stochastic gradient descent under convex and non-convex landscape. In this example, we provide non-asymptotic bounds that highly depend on the sparsity of the receptive field constructed by the algorithm. \textbf{(2)} The constrained regression problem on a 1-layer linear equivariant GNN. In this example, we present lower bounds for the discrepancy between the two types of stability, which justified the multi-fidelity design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑