arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.11600cs.PLcs.AIcs.LG
收藏

No Saved Kaleidoscope:一种具有Pythonic语法的100%即时编译神经网络编程语言

No Saved Kaleidosope: an 100% Jitted Neural Network Coding Language with Pythonic Syntax

Federal University of Technology - Paraná(巴拉那联邦技术大学);Federal University of Goias(戈亚斯联邦大学);São Paulo State University(圣保罗州立大学);Federal University of Mato Grosso(马托格罗索联邦大学)
Augusto Seben da Rosa, Marlon Daniel Angeli, Jorge Aikes Junior, Alef Iury Ferreira, Lucas Rafael Gris, Anderson da Silva Soares, Arnaldo Candido Junior, Freder… 展开作者

Augusto Seben da Rosa, Marlon Daniel Angeli, Jorge Aikes Junior, Alef Iury Ferreira, Lucas Rafael Gris, Anderson da Silva Soares, Arnaldo Candido Junior, Frederico Santos de Oliveira, Gabriel Trevisan Damke, Rafael Teixeira Sousa

AI总结:

本文提出一种基于C++、LLVM和Cuda的即时编译神经网络语言,具备Pythonic语法和自动微分,在CIFAR-10上性能与速度接近PyTorch,但ImageNet和GRU任务表现有差异。

摘要:

我们开发了一个用于训练人工神经网络的即时编译器,采用C++、LLVM和Cuda实现。该编译器具有面向对象特性、强类型、用于数据预处理的并行工作器、Pythonic表达式语法、类似PyTorch的模型声明以及自动微分。我们实现了缓存和池化机制以管理显存(VRAM),使用cuBLAS进行高性能矩阵乘法,使用cuDNN处理卷积层。我们在ImageNet上使用残差卷积神经网络进行实验,达到了相似的速度但性能有所下降。此外,GRU网络实验显示精度相似,但我们的编译器在该任务中速度有所下降。然而,我们的编译器在CIFAR-10基准测试中展现出令人鼓舞的结果,在该测试中我们达到了与PyTorch相同的性能和大致相同的速度。我们已将代码公开在:https://github.com/NoSavedDATA/NoSavedKaleidoscope

英文摘要:

We developed a jitted compiler for training Artificial Neural Networks using C++, LLVM and Cuda. It features object-oriented characteristics, strong typing, parallel workers for data pre-processing, pythonic syntax for expressions, PyTorch like model declaration and Automatic Differentiation. We implement the mechanisms of cache and pooling in order to manage VRAM, cuBLAS for high performance matrix multiplication and cuDNN for convolutional layers. Our experiments with Residual Convolutional Neural Networks on ImageNet, we reach similar speed but degraded performance. Also, the GRU network experiments show similar accuracy, but our compiler have degraded speed in that task. However, our compiler demonstrates promising results at the CIFAR-10 benchmark, in which we reach the same performance and about the same speed as PyTorch. We make the code publicly available at: https://github.com/NoSavedDATA/NoSavedKaleidoscope

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑