We introduce a new regression framework designed to deal with large-scale, complex data that lies around a low-dimensional manifold with noises. Our approach first constructs a graph representation, referred to as the skeleton, to capture the underlying geometric structure. We then define metrics on the skeleton graph and apply nonparametric regression techniques, along with feature transformations based on the graph, to estimate the regression function. We also discuss the limitations of some nonparametric regressors with respect to the general metric space such as the skeleton graph. The proposed regression framework suggests a novel way to deal with data with underlying geometric structures and provides additional advantages in handling the union of multiple manifolds, additive noises, and noisy observations. We provide statistical guarantees for the proposed method and demonstrate its effectiveness through simulations and real data examples.
Minimax-optimal semi-supervised regression on unknown manifolds
Weizmann Institute of Science(魏茨曼科学研究所)
我们考虑半监督回归时,预测变量是从一个未知的流形。解决这个问题的一个简单的两步方法是:(i)使用标记和未标记的实例来估计任何一对点之间的流形测地线距离;以及(ii)基于这些距离估计应用k最近邻回归量。我们证明了足够多的未标记的点,这种简单的方法测地线kNN回归达到最佳的有限样本极大极小界的均方误差,如果已知的流形。此外,我们展示了如何可以有效地实现这种方法,只需要O(k N log N)的操作来估计所有N个标记和未标记点的回归函数。我们在两个具有流形结构的数据集上说明了这种方法:使用WiFi指纹的室内定位和面部姿态估计。在这两种情况下,测地线kNN比流行的拉普拉斯特征向量回归更准确,更快。
Intrinsic Gaussian Process Regression Modeling for Manifold-valued Response Variable
International Institute of Finance, School of Management, University of Science and Technology of China(中国科学技术大学管理学院国际金融学院);Department of Statistics and Data Science, College of Science, Southern University of Science and Technology(南方科技大学理学院统计与数据科学系)
Technische Universität Darmstadt(达姆施塔特工业大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);University of Cambridge(剑桥大学);Imperial College London(帝国理工学院)
Active Learning for Manifold Gaussian Process Regression
Dept. of Applied Mathematics(应用数学系);Illinois Institute of Technology(伊利诺伊理工学院);Dept. of Mathematics and Statistics(数学与统计学系);University of Massachusetts Amherst(马萨诸塞大学阿姆赫斯特分校);Dept. of Mathematics(数学系);University of California Riverside(加州大学河滨分校)
Istituto Italiano di Tecnologia(意大利技术研究院);Imperial College London(伦敦帝国学院);Massachusetts Institute of Technology(麻省理工学院);Universitá degli Studi di Genova(热那亚大学)
Linear Regression on Manifold Structured Data: the Impact of Extrinsic Geometry on Solutions
University of Texas at Austin(德克萨斯大学奥斯汀分校);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Oden Institute for Computational Engineering and Sciences(奥登计算工程与科学研究所)
Georgia Institute of Technology(佐治亚理工学院);Johns Hopkins University(约翰斯·霍普金斯大学);University of Genova(热那亚大学)
本文考虑了$\mathbb{R}^D$上的估计函数的回归问题,但该估计函数是在具有$ d \ll D $的$d$维流形$ \mathcal{M} \subset \mathbb{R}^D $上支持的.借鉴多分辨率分析和非线性逼近的思想,我们在多个尺度上构建$\mathcal{M}$的低维坐标,并通过局部多项式拟合进行多尺度回归。我们提出了一个数据驱动的小波阈值方案,自动适应未知的规律性的功能,允许有效的估计功能表现出不均匀的规律性在不同的位置和尺度。我们分析了我们的方法的推广误差,证明有限样本界的高概率丰富类的先验。我们的估计达到最佳的学习率(对数因子),如果功能定义在一个已知的欧几里得域的尺寸$d$,而不是一个未知的流形嵌入在$\mathbb{R}^D$。所实现的算法具有准线性复杂度的样本量,常数线性的$D$和指数的$d$。因此,我们的工作建立了一个新的框架,回归低维集嵌入在高维,快速实现和强有力的理论保证。
Department of Physics, University of Massachusetts, Amherst, USA(马萨诸塞大学物理系);Department of Mathematics and Statistics, University of Massachusetts, Amherst, USA(马萨诸塞大学数学与统计学系);Humboldt University of Berlin , Berlin, Germany(柏林洪堡大学);Zuse Institute Berlin, Berlin, Germany(柏林祖布研究所)
Laboratoire de Mathématiques d’Orsay(奥赛数学实验室);Université Paris-Saclay(巴黎萨克雷大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);University of California(加利福尼亚大学);Laboratoire de Probabilités et Statistiques Mathématiques(数学概率与统计实验室);Université de Paris(巴黎大学)
MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency
Shandong University(山东大学);AICFVE, Beijing Film Academy(北京电影学院AICFVE);Tel-Aviv University(特拉维夫大学);University of Cyprus(塞浦路斯大学);RISE Research Centre(RISE研究中心);Edinburgh University(爱丁堡大学);The Hebrew University of Jerusalem(耶路撒冷希伯来大学);CFCS, Peking University(北京大学CFCS)
Learning Parametric Nitrogen Fertilizer Response Curves Using Neuro Symbolic Regression
Aston Centre for Artificial Intelligence Research and Application(阿斯顿人工智能研究与应用中心);Aston University(阿斯顿大学);Gianforte School of Computing(吉安福特计算学院);Montana State University(蒙大拿州立大学)
XpertAI: uncovering regression model strategies for sub-manifolds
Machine Learning Group, Technische Universität Berlin, Germany BIFOLD -- Berlin Institute for the Foundations of Learning;Data, Berlin, Germany Department of Artificial Intelligence, Korea University, Seoul, Korea Max Planck Institute for Informatics, Saarbrücken, Germany Charit\'e -- Universit\"atsmedizin Berlin, Germany