通过零泄漏重建路由和自主任务发现实现模块化持续学习
Modular Continual Learning via Zero-Leakage Reconstruction Routing and Autonomous Task Discovery
AI总结：
本文提出一种硅本征模块化架构，通过任务特定专家和基于异常值的门卫实现结构参数隔离，采用同时流水线并行处理教师学习、学生蒸馏和路由流形获取，有效提升计算效率并满足隐私要求。
摘要：
灾难性遗忘仍然是人工神经网络序列任务学习中的主要障碍。我们提出了一种硅本征模块化架构，通过任务特定专家和分布式、基于异常值的门卫实现结构参数隔离。超越传统顺序整合，我们的框架利用同时流水线，使教师学习、学生蒸馏和路由流形获取在数据本地训练会话中并行进行。此方法确保计算效率，并通过在任务学习后立即删除原始数据来遵守隐私要求如GDPR。我们证明了紧瓶颈自编码器（TB-AE）可以有效区分高维潜在空间中语义拥挤的流形，克服标准变分方法固有的后验崩溃。通过建立严格的拓扑边界，我们的TB-AE在4096-D LLM嵌入中解决潜在空间拥挤，提供稳健的无监督新颖性信号。此外，我们验证了自主检索机制，能够自信地识别返回的流形，使稳定的一生学习无需冗余模块实例化。实证结果表明，我们的“实时蒸馏”方法作为自然正则化器，在计算机视觉和自然语言处理领域实现强大的保留，而不会遭受学生保真度差距。
英文摘要：
Catastrophic forgetting remains a primary hurdle in sequential task learning for artificial neural networks. We propose a silicon-native modular architecture that achieves structural parameter isolation using Task-Specific Experts and a distributed, outlier-based Gatekeeper. Moving beyond traditional sequential consolidation, our framework utilizes a Simultaneous Pipeline where Teacher learning, Student distillation, and Router manifold acquisition occur in parallel while raw data is present in a localized training session. This approach ensures computational efficiency and complies with privacy mandates like GDPR by deleting raw data as soon as a task is learned. We demonstrate that a Tight-Bottleneck Autoencoder (TB-AE) can effectively distinguish semantically crowded manifolds in high-dimensional latent spaces, overcoming the posterior collapse inherent to standard variational methods. By establishing strict topological boundaries, our TB-AE resolves latent space crowding in 4096-D LLM embeddings to provide a robust, unsupervised novelty signal. Furthermore, we validate an Autonomous Retrieval mechanism that confidently identifies returning manifolds, enabling stable lifelong learning without redundant module instantiation. Empirical results demonstrate that our ``Live Distillation'' approach acts as a natural regularizer, achieving strong retention across computer vision and natural language processing domains without suffering a student fidelity gap.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
