arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.02166cs.LGcs.AI
收藏

深度权重空间中推理过程的动态神经图编码

Dynamic Neural Graph Encoding of Inference Processes in Deep Weight Space

University of Toronto(多伦多大学);National University of Singapore(新加坡国立大学);McMaster University(麦马斯特大学);Concordia University(康科迪亚大学)

Di Wu, Huan Liu, Zhixiang Chi, Yuanhao Yu, Konstantinos N. Plataniotis, Yang Wang

AI总结:

提出动态神经图编码器(DNG-Encoder),通过动态图表示神经网络参数并保留推理的时序特性,在INR分类任务上比现有方法提升约10%准确率。

摘要:

使用神经网络作为隐式数据表示的快速进展引起了人们对开发分析和处理其他神经网络权重空间的机器学习方法的浓厚兴趣。然而,高效处理这些高维权重空间仍然具有挑战性。现有方法通常忽略神经网络推理中层逐层处理的顺序性质。在这项工作中,我们提出了一种使用动态图表示神经网络参数的新方法,捕捉推理的时间动态。我们的动态神经图编码器(DNG-Encoder)处理这些图,保留了神经处理的顺序性质。此外,我们还利用DNG-Encoder开发了INR2JLS(隐式神经表示到联合潜在空间),以促进下游应用,例如分类隐式神经表示(INR)。我们的方法在多个任务上展示了显著改进,在CIFAR-100-INR上比最先进的INR分类准确率提高了约10%。

英文摘要:

The rapid advancements in using neural networks as implicit data representations have attracted significant interest in developing machine learning methods that analyze and process the weight spaces of other neural networks. However, efficiently handling these highdimensional weight spaces remains challenging. Existing methods often overlook the sequential nature of layer-by-layer processing in neural network inference. In this work, we propose a novel approach using dynamic graphs to represent neural network parameters, capturing the temporal dynamics of inference. Our Dynamic Neural Graph Encoder (DNG-Encoder) processes these graphs, preserving the sequential nature of neural processing. Additionally, we also leverage DNG-Encoder to develop INR2JLS (Implicit Neural Representation to Joint Latent Space) for facilitate downstream applications, such as classifying Implicit Neural Representations (INRs). Our approach demonstrates significant improvements across multiple tasks, surpassing the state-of-the-art INR classification accuracy by approximately 10% on the CIFAR-100-INR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑