25 pages. Modified previous material for greater clarity; added new material about approximate Lipschitz constraints, Approximate Lipschitz Embedding (ALE), and uniform convergence; added material on constrained minimization of raw stress to the appendix
Multidimensional scaling (MDS) is the act of embedding proximity information about a set of $n$ objects in $d$-dimensional Euclidean space. As originally conceived by the psychometric community, MDS was concerned with embedding a fixed set of proximities associated with a fixed set of objects. Modern concerns, e.g., that arise in developing asymptotic theories for statistical inference on random graphs, more typically involve studying the limiting behavior of a sequence of proximities associated with an increasing set of objects. Here we are concerned with embedding dissimilarities by minimizing Kruskal's (1964) raw stress criterion. Standard results from the theory of point-to-set maps can be used to establish that, if $n$ is fixed and a sequence of dissimilarity matrices converges, then the limit of their embedded structures is the embedded structure of the limiting dissimilarity matrix. But what if $n$ increases? It then becomes necessary to reformulate MDS so that the entire sequence of embedding problems can be viewed as a sequence of optimization problems in a fixed space. We present such a reformulation, {\em continuous MDS}. Within the continuous MDS framework, we derive two $L^p$ consistency results, one for embedding without constraints on the configuration, the other for embedding subject to {\em approximate Lipschitz constraints}\/ that encourage smoothness of the embedding function. The latter approach, {\em Approximate Lipschitz Embedding}\/ (ALE) is new. Finally, we demonstrate that embedded structures produced by ALE can be interpolated in a way that results in uniform convergence.
DinTucker: Scaling up Gaussian process models on multidimensional arrays with billions of elements
Purdue University(普渡大学);IBM T. J. Watson Research Center(IBM T·J·沃森研究中心)
无限Tucker分解(InfTucker)和随机函数先验模型作为无限可交换阵列上的非参数贝叶斯模型,是比广泛使用的多线性分解方法(包括Tucker和PARAFAC分解)更强大的模型,(部分)由于它们能够建模阵列元素之间的非线性关系。尽管它们具有很好的预测性能和良好的理论基础,但由于训练时间过长,它们无法处理大量数据。为了克服这一限制,我们提出了分布式无限塔克(DINTUCKER),一个大规模的非线性张量分解算法的MAPREDUCE。在保持InfTucker的预测准确性的同时,它可以在海量数据上进行扩展。DINTUCKER基于一种新的分层贝叶斯模型,可以在子阵列上对InfTucker进行本地训练,并从所有本地训练结果中集成信息。我们使用分布式随机梯度下降,再加上变分推理,训练这个模型。我们将DINTUCKER应用于来自“Read the Web”项目(Carlson et al.,2010)和信息安全领域,并将其与最先进的大规模张量分解方法GigaTensor进行比较。在这两个数据集上,DINTUCKER以更少的计算时间实现了更高的预测精度。