arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2404.09613cs.ETcs.AIcs.AR
收藏

基于阻变存储器的高效精准神经场重建

Efficient and accurate neural field reconstruction using resistive memory

the University of Hong Kong(香港大学);ACCESS – AI Chip Center for Emerging Smart Systems, InnoHK Centers(ACCESS——新兴智能系统AI芯片中心(InnoHK中心));Institute of Mind, the University of Hong Kong(香港大学心智研究所);Institute of Microelectronics, Chinese Academy of Sciences(中国科学院微电子研究所);University of Chinese Academy of Sciences(中国科学院大学)
Yifei Yu, Shaocong Wang, Woyu Zhang, Xinyuan Zhang, Xiuzhe Wu, Yangu He, Jichang Yang, Yue Zhang, Ning Lin, Bo Wang, Xi Chen, Songqi Wang, Xumeng Zhang, Xiaojua… 展开作者

Yifei Yu, Shaocong Wang, Woyu Zhang, Xinyuan Zhang, Xiuzhe Wu, Yangu He, Jichang Yang, Yue Zhang, Ning Lin, Bo Wang, Xi Chen, Songqi Wang, Xumeng Zhang, Xiaojuan Qi, Zhongrui Wang, Dashan Shang, Qi Liu, Kwang-Ting Cheng, Ming Liu

AI总结:

针对传统信号重建的软硬件瓶颈,该研究提出软硬件协同优化方案:软件端用低秩分解与结构化剪枝压缩神经场,硬件端设计基于阻变存储器的存内计算平台,在多任务中实现能效与并行性大幅提升且不损失重建质量。

摘要:

人类通过将稀疏观测整合到大规模互连的突触与神经元中来构建空间感知,具备卓越的并行性与效率。在AI中复刻这一能力在医学成像、AR/VR以及具身AI领域有广泛应用,这些场景的输入数据通常较为稀疏且计算资源有限。然而,数字计算机上的传统信号重建方法同时面临软件与硬件层面的挑战。在软件方面,传统显式信号表示存在存储效率低下的问题。硬件层面的障碍包括限制CPU与内存间数据传输的冯·诺依曼瓶颈,以及CMOS电路在支持并行处理方面的局限。我们提出了一种软硬件协同优化的系统化方法,用于从稀疏输入中重建信号。软件层面,我们采用神经场(neural field)通过神经网络隐式表示信号,并进一步通过低秩分解与结构化剪枝对其进行压缩。硬件层面,我们设计了一个基于阻变存储器的存内计算(computing-in-memory, CIM)平台,该平台包含高斯编码器(Gaussian Encoder, GE)与MLP处理引擎(Processing Engine, PE)。高斯编码器利用阻变存储器的固有随机性实现高效的输入编码,而MLP处理引擎则通过硬件感知量化(Hardware-Aware Quantization, HAQ)电路实现精准的权重映射。我们在一款40nm工艺、256Kb容量的阻变存储器存内计算宏单元上验证了该系统的有效性,在3D CT稀疏重建、新视角合成以及动态场景新视角合成等任务中,在不损失重建质量的前提下,实现了能效与并行性的大幅提升。这项工作推动了AI驱动的信号修复技术发展,为未来高效稳健的医学AI与3D视觉应用铺平了道路。

英文摘要:

Human beings construct perception of space by integrating sparse observations into massively interconnected synapses and neurons, offering a superior parallelism and efficiency. Replicating this capability in AI finds wide applications in medical imaging, AR/VR, and embodied AI, where input data is often sparse and computing resources are limited. However, traditional signal reconstruction methods on digital computers face both software and hardware challenges. On the software front, difficulties arise from storage inefficiencies in conventional explicit signal representation. Hardware obstacles include the von Neumann bottleneck, which limits data transfer between the CPU and memory, and the limitations of CMOS circuits in supporting parallel processing. We propose a systematic approach with software-hardware co-optimizations for signal reconstruction from sparse inputs. Software-wise, we employ neural field to implicitly represent signals via neural networks, which is further compressed using low-rank decomposition and structured pruning. Hardware-wise, we design a resistive memory-based computing-in-memory (CIM) platform, featuring a Gaussian Encoder (GE) and an MLP Processing Engine (PE). The GE harnesses the intrinsic stochasticity of resistive memory for efficient input encoding, while the PE achieves precise weight mapping through a Hardware-Aware Quantization (HAQ) circuit. We demonstrate the system's efficacy on a 40nm 256Kb resistive memory-based in-memory computing macro, achieving huge energy efficiency and parallelism improvements without compromising reconstruction quality in tasks like 3D CT sparse reconstruction, novel view synthesis, and novel view synthesis for dynamic scenes. This work advances the AI-driven signal restoration technology and paves the way for future efficient and robust medical AI and 3D vision applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑