arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.06141cs.LGcs.AIphysics.optics
收藏

光子神经网络的对偶自适应训练

Dual adaptive training of photonic neural networks

Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心)

Ziyang Zheng, Zhengyang Duan, Hang Chen, Rui Yang, Sheng Gao, Haiou Zhang, Hongkai Xiong, Xing Lin

摘要:

光子神经网络(PNN)是一种出色的模拟人工智能(AI)加速器,它使用光子代替电子进行计算,具有低延迟、高能效和高并行性的特点。然而,现有的训练方法不能解决大规模PNN中系统误差的广泛积累,导致物理系统中模型性能的显着下降。在这里,我们提出了双重自适应训练(DAT),允许PNN模型适应大量的系统误差,并在部署过程中保持其性能。通过引入任务-相似度联合优化的系统误差预测网络,DAT实现了PNN数值模型与物理系统之间的高相似度映射,以及双反向传播训练过程中的高精度梯度计算。我们通过使用衍射PNN和基于干涉的PNN对图像分类任务进行验证DAT的有效性。DAT在主要系统误差下成功训练了大规模PNN,并保持了与无误差系统相当的模型分类精度。结果进一步证明了其优于最先进的原位训练方法的性能。DAT为构建大规模PNN以实现高级架构提供了关键支持,并且可以推广到具有模拟计算错误的其他类型的AI系统。

英文摘要:

Photonic neural network (PNN) is a remarkable analog artificial intelligence (AI) accelerator that computes with photons instead of electrons to feature low latency, high energy efficiency, and high parallelism. However, the existing training approaches cannot address the extensive accumulation of systematic errors in large-scale PNNs, resulting in a significant decrease in model performance in physical systems. Here, we propose dual adaptive training (DAT) that allows the PNN model to adapt to substantial systematic errors and preserves its performance during the deployment. By introducing the systematic error prediction networks with task-similarity joint optimization, DAT achieves the high similarity mapping between the PNN numerical models and physical systems and high-accurate gradient calculations during the dual backpropagation training. We validated the effectiveness of DAT by using diffractive PNNs and interference-based PNNs on image classification tasks. DAT successfully trained large-scale PNNs under major systematic errors and preserved the model classification accuracies comparable to error-free systems. The results further demonstrated its superior performance over the state-of-the-art in situ training approaches. DAT provides critical support for constructing large-scale PNNs to achieve advanced architectures and can be generalized to other types of AI systems with analog computing errors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑