The volume of the distribution of weight sets associated with a loss value may be the source of implicit regularization from overparameterization due to the phenomenon of contracting volume with increasing dimensions for geometric figures demonstrated by hyperspheres. We introduce the geometric regularization conjecture and extract to an explanation for the double descent phenomenon by considering a similar property resulting from shrinking intrinsic dimensionality of the distribution of potential weight set updates available along training path, where if that distribution retracts across a volume verses dimensionality curve peak when approaching the global minima we could expect geometric regularization to re-emerge. We illustrate how data fidelity representational complexity may influence model capacity double descent interpolation thresholds. The existence of epoch and model capacity double descent curves originating from different geometric forms may imply universality of closed n-manifolds having dimensionally adjusted n-sphere volumetric correspondence.
Solving Inverse Problems with Deep Linear Neural Networks: Global Convergence Guarantees for Gradient Descent with Weight Decay
Department of Mathematics, Technical University of Munich, Munich Center for Machine Learning (MCML), Munich, Germany（数学系，慕尼黑技术大学，慕尼黑机器学习中心（MCML），慕尼黑，德国）；Committee on Computational and Applied Mathematics, University of Chicago, Chicago, IL（计算与应用数学委员会，芝加哥大学，芝加哥，伊利诺伊）；Department of Electrical & Computer Engineering, University of Washington, Seattle, WA（电气与计算机工程系，华盛顿大学，西雅图，华盛顿）；Departments of Statistics and Computer Science, University of Chicago, Chicago, IL（统计学与计算机科学系，芝加哥大学，芝加哥，伊利诺伊）
Diverse via bounded Agreement: Geometric Regularization for Multimodal Fusion
Department of Informatics University of Bern（伯尔尼大学信息学院）；College of Computing and Data Science Nanyang Technological University（南洋理工大学计算机与数据科学学院）；School of Software Engineering Xi’an Jiaotong University（西安交通大学软件工程学院）
Iterate averaging as regularization for stochastic gradient descent
Universitat Pompeu Fabra（庞培法布拉大学）；Massachusetts Institute of Technology（麻省理工学院）；Istituto Italiano di Tecnologia（意大利理工学院）；Università degli Studi di Genova（热那亚大学）
