P2M：一种面向资源受限TinyML应用的内存中像素处理范例
P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications
摘要：
处理由最先进的高分辨率摄像头生成的大量数据的需求激发了新型节能的设备上AI解决方案。这种相机中的视觉数据通常由传感器像素阵列以模拟电压的形式捕获，然后使用模数转换器（ADC）转换到数字域以进行后续AI处理。最近的研究试图以近传感器和传感器内处理的形式利用大规模并行低功耗模拟/数字计算，其中AI计算部分在像素阵列的外围执行，部分在单独的板载CPU/加速器中执行。不幸的是，高分辨率的输入图像仍然需要在相机和AI处理单元之间逐帧传输，这会导致能源、带宽和安全瓶颈。为了缓解这个问题，我们提出了一种新的内存中像素处理（P2 M）范式，通过添加对模拟多通道、多位卷积、批量归一化和ReLU（整流线性单元）的支持来定制像素阵列。我们的解决方案包括一个整体的算法电路协同设计方法，由此产生的P2 M范式可以用作嵌入内存密集型卷积神经网络（CNN）模型的前几层的替代品，在可代工制造的CMOS图像传感器平台。我们的实验结果表明，P2 M将传感器和模数转换的数据传输带宽降低了约21倍，与标准近处理或传感器内实现相比，在TinyML用例上处理MobileNetV 2模型时产生的能量延迟积（EDP）可达约11倍，而测试准确性没有任何显着下降。
英文摘要：
The demand to process vast amounts of data generated from state-of-the-art high resolution cameras has motivated novel energy-efficient on-device AI solutions. Visual data in such cameras are usually captured in the form of analog voltages by a sensor pixel array, and then converted to the digital domain for subsequent AI processing using analog-to-digital converters (ADC). Recent research has tried to take advantage of massively parallel low-power analog/digital computing in the form of near- and in-sensor processing, in which the AI computation is performed partly in the periphery of the pixel array and partly in a separate on-board CPU/accelerator. Unfortunately, high-resolution input images still need to be streamed between the camera and the AI processing unit, frame by frame, causing energy, bandwidth, and security bottlenecks. To mitigate this problem, we propose a novel Processing-in-Pixel-in-memory (P2M) paradigm, that customizes the pixel array by adding support for analog multi-channel, multi-bit convolution, batch normalization, and ReLU (Rectified Linear Units). Our solution includes a holistic algorithm-circuit co-design approach and the resulting P2M paradigm can be used as a drop-in replacement for embedding memory-intensive first few layers of convolutional neural network (CNN) models within foundry-manufacturable CMOS image sensor platforms. Our experimental results indicate that P2M reduces data transfer bandwidth from sensors and analog to digital conversions by ~21x, and the energy-delay product (EDP) incurred in processing a MobileNetV2 model on a TinyML use case for visual wake words dataset (VWW) by up to ~11x compared to standard near-processing or in-sensor implementations, without any significant drop in test accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
