NeuSemSlice:通过神经元级语义切片实现有效的 DNN 模型维护
NeuSemSlice: Towards Effective DNN Model Maintenance via Neuron-level Semantic Slicing
AI总结:
本文提出 NeuSemSlice 框架,利用神经元级语义切片识别、分类并合并关键神经元,并配套重构、重适配与增量开发策略,在三类 DNN 维护任务上显著优于基线。
摘要:
深度神经网络(DNN)被广泛应用于众多学科,其特征是集成且单体式的架构,这使其有别于传统软件系统。这种架构差异给维护任务带来了特殊挑战,例如模型重构(如模型压缩)、重新适配(如拟合新样本)和增量开发(如持续知识积累)。先前研究通过识别任务关键的神经元层,并将神经网络划分为语义相似的顺序模块来应对这些挑战。然而,此类层级方法无法精确识别和操纵神经元级语义组件,限制了其在更细粒度模型维护任务中的适用性。在本工作中,我们实现了 NeuSemSlice,这是一个新颖框架,引入语义切片技术以有效识别 DNN 模型中关键的神经元级语义组件,用于语义感知的模型维护任务。具体而言,语义切片根据关键神经元的语义相似性,对跨不同类别和层的关键神经元进行识别、分类和合并,从而使其在后续任务中具备灵活性和有效性。针对语义感知的模型维护任务,我们基于语义切片提供了一系列新颖策略以增强 NeuSemSlice,包括用于模型重构的语义组件(即关键神经元)保留、用于模型重新适配的关键神经元调优,以及用于模型增量开发的非关键神经元训练。全面评估表明,NeuSemSlice 在所有三项任务中均显著优于基线方法。
英文摘要:
Deep Neural networks (DNNs), extensively applied across diverse disciplines, are characterized by their integrated and monolithic architectures, setting them apart from conventional software systems. This architectural difference introduces particular challenges to maintenance tasks, such as model restructure (e.g., model compression), re-adaptation (e.g., fitting new samples), and incremental development (e.g., continual knowledge accumulation). Prior research addresses these challenges by identifying task-critical neuron layers, and dividing neural networks into semantically-similar sequential modules. However, such layer-level approaches fail to precisely identify and manipulate neuron-level semantic components, restricting their applicability to finer-grained model maintenance tasks. In this work, we implement NeuSemSlice, a novel framework that introduces the semantic slicing technique to effectively identify critical neuron-level semantic components in DNN models for semantic-aware model maintenance tasks. Specifically, semantic slicing identifies, categorizes and merges critical neurons across different categories and layers according to their semantic similarity, enabling their flexibility and effectiveness in the subsequent tasks. For semantic-aware model maintenance tasks, we provide a series of novel strategies based on semantic slicing to enhance NeuSemSlice. They include semantic components (i.e., critical neurons) preservation for model restructure, critical neuron tuning for model re-adaptation, and non-critical neuron training for model incremental development. A thorough evaluation has demonstrated that NeuSemSlice significantly outperforms baselines in all three tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。