No Saved Kaleidoscope:一种具有Pythonic语法的100%即时编译神经网络编程语言
No Saved Kaleidosope: an 100% Jitted Neural Network Coding Language with Pythonic Syntax
AI总结:
本文提出一种基于C++、LLVM和Cuda的即时编译神经网络语言,具备Pythonic语法和自动微分,在CIFAR-10上性能与速度接近PyTorch,但ImageNet和GRU任务表现有差异。
摘要:
我们开发了一个用于训练人工神经网络的即时编译器,采用C++、LLVM和Cuda实现。该编译器具有面向对象特性、强类型、用于数据预处理的并行工作器、Pythonic表达式语法、类似PyTorch的模型声明以及自动微分。我们实现了缓存和池化机制以管理显存(VRAM),使用cuBLAS进行高性能矩阵乘法,使用cuDNN处理卷积层。我们在ImageNet上使用残差卷积神经网络进行实验,达到了相似的速度但性能有所下降。此外,GRU网络实验显示精度相似,但我们的编译器在该任务中速度有所下降。然而,我们的编译器在CIFAR-10基准测试中展现出令人鼓舞的结果,在该测试中我们达到了与PyTorch相同的性能和大致相同的速度。我们已将代码公开在:https://github.com/NoSavedDATA/NoSavedKaleidoscope
英文摘要:
We developed a jitted compiler for training Artificial Neural Networks using C++, LLVM and Cuda. It features object-oriented characteristics, strong typing, parallel workers for data pre-processing, pythonic syntax for expressions, PyTorch like model declaration and Automatic Differentiation. We implement the mechanisms of cache and pooling in order to manage VRAM, cuBLAS for high performance matrix multiplication and cuDNN for convolutional layers. Our experiments with Residual Convolutional Neural Networks on ImageNet, we reach similar speed but degraded performance. Also, the GRU network experiments show similar accuracy, but our compiler have degraded speed in that task. However, our compiler demonstrates promising results at the CIFAR-10 benchmark, in which we reach the same performance and about the same speed as PyTorch. We make the code publicly available at: https://github.com/NoSavedDATA/NoSavedKaleidoscope
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。