用于持续聚类的前向-后向知识蒸馏
Forward-Backward Knowledge Distillation for Continual Clustering
AI总结：
本文提出无监督持续聚类（UCC）概念及前向-后向知识蒸馏方法（FBCC），通过教师-学生模型的两阶段知识蒸馏有效缓解灾难性遗忘，在多任务聚类中实现更优性能与内存效率。
摘要：
无监督持续学习（UCL）是机器学习中一个新兴领域，致力于使神经网络在没有显式标签信息的情况下顺序学习任务。灾难性遗忘（CF），即模型在学习新任务时会忘记之前学过的任务，在持续学习中构成重大挑战，在无法获取数据标签信息的UCL中尤为严重。CF缓解策略，如知识蒸馏和重放缓冲区，通常面临内存效率低下和隐私问题。尽管目前UCL的研究已致力于优化数据表示并处理流数据背景下的CF，但明显缺乏专门针对无监督聚类设计的算法。为填补这一空白，本文引入了无监督持续聚类（UCC）的概念。我们提出了用于无监督持续聚类的前向-后向知识蒸馏（FBCC），以在UCC背景下对抗CF。FBCC采用一个带有聚类投影器的单一持续学习者（“教师”），以及多个学生模型，来解决CF问题。该方法包含两个阶段：前向知识蒸馏，教师模型在专业学生模型的指导下学习新聚类，同时保留先前任务的知识；后向知识蒸馏，学生模型模仿教师行为以保留特定任务知识，协助教师完成后续任务。FBCC标志着UCC的先驱方法，在跨多种任务的聚类中展现出增强的性能和内存效率，优于将聚类算法应用于最先进UCL算法的潜在空间。
英文摘要：
Unsupervised Continual Learning (UCL) is a burgeoning field in machine learning, focusing on enabling neural networks to sequentially learn tasks without explicit label information. Catastrophic Forgetting (CF), where models forget previously learned tasks upon learning new ones, poses a significant challenge in continual learning, especially in UCL, where labeled information of data is not accessible. CF mitigation strategies, such as knowledge distillation and replay buffers, often face memory inefficiency and privacy issues. Although current research in UCL has endeavored to refine data representations and address CF in streaming data contexts, there is a noticeable lack of algorithms specifically designed for unsupervised clustering. To fill this gap, in this paper, we introduce the concept of Unsupervised Continual Clustering (UCC). We propose Forward-Backward Knowledge Distillation for unsupervised Continual Clustering (FBCC) to counteract CF within the context of UCC. FBCC employs a single continual learner (the ``teacher'') with a cluster projector, along with multiple student models, to address the CF issue. The proposed method consists of two phases: Forward Knowledge Distillation, where the teacher learns new clusters while retaining knowledge from previous tasks with guidance from specialized student models, and Backward Knowledge Distillation, where a student model mimics the teacher's behavior to retain task-specific knowledge, aiding the teacher in subsequent tasks. FBCC marks a pioneering approach to UCC, demonstrating enhanced performance and memory efficiency in clustering across various tasks, outperforming the application of clustering algorithms to the latent space of state-of-the-art UCL algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
