Rendering and inverse-rendering algorithms that drive conventional computer graphics have recently been superseded by neural representations (NR). NRs have recently been used to learn the geometric and the material properties of the scenes and use the information to synthesize photorealistic imagery, thereby promising a replacement for traditional rendering algorithms with scalable quality and predictable performance. In this work we ask the question: Does neural graphics (NG) need hardware support? We studied representative NG applications showing that, if we want to render 4k res. at 60FPS there is a gap of 1.5X-55X in the desired performance on current GPUs. For AR/VR applications, there is an even larger gap of 2-4 OOM between the desired performance and the required system power. We identify that the input encoding and the MLP kernels are the performance bottlenecks, consuming 72%,60% and 59% of application time for multi res. hashgrid, multi res. densegrid and low res. densegrid encodings, respectively. We propose a NG processing cluster, a scalable and flexible hardware architecture that directly accelerates the input encoding and MLP kernels through dedicated engines and supports a wide range of NG applications. We also accelerate the rest of the kernels by fusing them together in Vulkan, which leads to 9.94X kernel-level performance improvement compared to un-fused implementation of the pre-processing and the post-processing kernels. Our results show that, NGPC gives up to 58X end-to-end application-level performance improvement, for multi res. hashgrid encoding on average across the four NG applications, the performance benefits are 12X,20X,33X and 39X for the scaling factor of 8,16,32 and 64, respectively. Our results show that with multi res. hashgrid encoding, NGPC enables the rendering of 4k res. at 30FPS for NeRF and 8k res. at 120FPS for all our other NG applications.
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights
School of Computing Informatics and Decision Systems Engineering at Arizona State University(亚利桑那州立大学计算、信息科学与工程学院);Computer Science and Artificial Intelligence Laboratory at Massachusetts Institute of Technology(麻省理工学院计算机科学与人工智能实验室);School of Computer Science at University of California, Los Angeles(加利福尼亚大学洛杉矶分校计算机科学学院);Parallel Computing Lab at Intel Labs(英特尔实验室并行计算实验室)
Hardware Acceleration for Third-Generation FHE and PSI Based on It
随着云服务的扩展,由于在计算过程中未加密的数据暴露给服务器,因此对用户数据隐私的严重关注。各种安全原语正在研究中,以在评估私有数据时保护隐私,包括全同态加密(FHE)、私有集合相交(PSI)等。然而,这些原语的禁止处理时间阻碍了它们的实际应用。这项工作提出并实现了一种使用Amazon Web Services(AWS)云FPGA加速第三代FHE的架构,标志着第一个第三代FHE硬件加速解决方案。我们还介绍了一种新的不平衡PSI协议的基础上第三代FHE,建议的硬件架构进行了优化。几个算法架构的协同优化技术,以允许通信和计算成本是独立的?的集合大小。测量结果表明,所提出的加速器实现了$>21\times$性能的改善相比,第三代FHE和建议PSI的各种关键子程序的软件实现。
Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processing
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室);School of Future Technology, University of Chinese Academy of Sciences(中国科学院大学未来技术学院)
MC$^2$A: Enabling Algorithm-Hardware Co-Design for Efficient Markov Chain Monte Carlo Acceleration
Department of Electrical Engineering - MICAS, KU Leuven, Belgium(电气工程系-MICAS,卢森堡大学);Department of Computer Science - DTAI, KU Leuven, Belgium(计算机科学系-DTAI,卢森堡大学);Department of Electronics and Nanoengineering, Aalto University(电子与纳米工程系,阿alto大学)
FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive Review
Key Laboratory of Low-Dimensional Quantum Structures and Quantum Control of Ministry of Education, School of Physics and Electronics, Hunan Normal University(教育部低维量子结构与量子控制重点实验室,物理电子学院,湖南师范大学)
Computer Systems Lab, Cornell University(康奈尔大学计算机系统实验室);Systems and Networking Research Group, Microsoft Research(微软研究院系统与网络研究组);Department of Electrical and Electronic Engineering, Imperial College London(伦敦帝国理工学院电子与电气工程系)
To Spike or Not To Spike: A Digital Hardware Perspective on Deep Learning Acceleration
Politecnico di Torino(都灵理工大学);Delft University of Technology(代尔夫特理工大学);University of Zürich and ETH Zürich(苏黎世大学和苏黎世联邦理工学院);University of California Santa Cruz(加州大学圣克鲁兹分校)
MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration
Integrated Circuits Lab, Electrical and Computer Engineering, Democritus University of Thrace (DUTH), Greece(德摩克利特大学特拉克分校集成电路实验室,电气与计算机工程,德摩克利特大学特拉克分校(DUTH),希腊)
A Reconfigurable Framework for AI-FPGA Agent Integration and Acceleration
Purdue University(普渡大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Moratuwa(莫拉图瓦大学);Stanford University(斯坦福大学);Istinye University(伊斯提耶大学)
Hardware acceleration for ultra-fast Neural Network training on FPGA for MRF map reconstruction
National Institute of Nuclear Physics (INFN), Division of Bologna(国家核物理研究所(INFN),博洛尼亚分校);National Institute of Nuclear Physics (INFN), Division of Pisa(国家核物理研究所(INFN),比萨分校)