带有增量 k-Means++ 初始化的 k-Medoids 聚类的精心初始化

InRank：增量低秩学习

InRank: Incremental Low-Rank Learning

本文提出增量低秩学习算法InRank，通过显式表示累积权重更新为低秩矩阵并逐步增加秩，在保持预测性能的同时显著降低了训练时间和模型大小。

Incremental Training of Deep Convolutional Neural Networks

We propose an incremental training method that partitions the original network into sub-networks, which are then gradually incorporated in the running network during the training process. To allow for a smooth dynamic growth of the network, we introduce a look-ahead initialization that outperforms the random initialization. We demonstrate that our incremental approach reaches the reference network baseline accuracy. Additionally, it allows to identify smaller partitions of the original state-of-the-art network, that deliver the same final accuracy, by using only a fraction of the global number…

The Implicit Bias of Depth: How Incremental Learning Drives Generalization

A leading hypothesis for the surprising generalization of neural networks is that the dynamics of gradient descent bias the model towards simple solutions, by searching through the solution space in an incremental order of complexity. We formally define the notion of incremental learning dynamics and derive the conditions on depth and initialization for which this phenomenon arises in deep linear models. Our main theoretical contribution is a dynamical depth separation result, proving that while shallow models can exhibit incremental learning dynamics, they require the initialization to be exp…

理解过参数化矩阵分解上梯度流闭式解的增量学习

Understanding Incremental Learning with Closed-form Solution to Gradient Flow on Overparamerterized Matrix Factorization

本文利用Riccati型矩阵微分方程的闭式解，定量解释对称过参数化矩阵分解中梯度流的增量学习机制，揭示其源于不同分量动力学的时间尺度分离。

MaxMin Linear Initialization for Fuzzy C-Means

Clustering is an extensive research area in data science. The aim of clustering is to discover groups and to identify interesting patterns in datasets. Crisp (hard) clustering considers that each data point belongs to one and only one cluster. However, it is inadequate as some data points may belong to several clusters, as is the case in text categorization. Thus, we need more flexible clustering. Fuzzy clustering methods, where each data point can belong to several clusters, are an interesting alternative. Yet, seeding iterative fuzzy algorithms to achieve high quality clustering is an issue.…

改进的k-means离群点鲁棒初始化方法

Improved Outlier Robust Seeding for k-means

针对k-means对离群点敏感的问题，提出D²采样的鲁棒变体，在常数比例离群点下达到O(1)近似，时间O(ndk)，输出O(k)簇，优于现有鲁棒方法。

iNeMo：面向鲁棒类别增量学习的增量神经网格模型

iNeMo: Incremental Neural Mesh Models for Robust Class-Incremental Learning

针对现有持续学习方法难以应对分布外场景的问题，提出可随时间扩展的增量神经网格模型iNeMo，搭配隐空间初始化策略与位置正则化项，在分类任务的域内和OOD场景下均优于基线，还实现了首个姿态估计增量学习方法。

Convergence Analysis and Implicit Regularization of Feedback Alignment for Deep Linear Networks

We theoretically analyze the Feedback Alignment (FA) algorithm, an efficient alternative to backpropagation for training neural networks. We provide convergence guarantees with rates for deep linear networks for both continuous and discrete dynamics. Additionally, we study incremental learning phenomena for shallow linear networks. Interestingly, certain specific initializations imply that negligible components are learned before the principal ones, thus potentially negatively affecting the effectiveness of such a learning algorithm; a phenomenon we classify as implicit anti-regularization. We…

Data-Driven Clustering via Parameterized Lloyd's Families

Algorithms for clustering points in metric spaces is a long-studied area of research. Clustering has seen a multitude of work both theoretically, in understanding the approximation guarantees possible for many objective functions such as k-median and k-means clustering, and experimentally, in finding the fastest algorithms and seeding procedures for Lloyd's algorithm. The performance of a given clustering algorithm depends on the specific application at hand, and this may not be known up front. For example, a "typical instance" may vary depending on the application, and different clustering he…

基于动态稀疏训练的持续学习：探索有效模型更新算法

Continual Learning with Dynamic Sparse Training: Exploring Algorithms for Effective Model Updates

本文首次实证研究动态稀疏训练各组件在持续学习中的作用，在CIFAR100和miniImageNet上分析初始化、增长策略与自适应性，发现不同稀疏度下最优配置不同，自适应性有助于提升持续学习性能。

Global $k$-means$++$：global $k$-means 聚类算法的一种有效松弛

Global $k$-means$++$: an effective relaxation of the global $k$-means clustering algorithm

针对 global $k$-means 计算成本高的问题，提出 global $k$-means$++$，利用 $k$-means$++$ 的中心选择概率降低计算量，并在多个基准上取得质量与速度俱佳的聚类效果。