发表机构
University of Edinburgh（爱丁堡大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种面向仅640KB SRAM微控制器的紧凑型深度学习超分辨率架构，对VL53L9CX SPAD传感器获取的54×42深度和强度图像进行×4上采样，首次在裸机MCU上实现片上SR推理。
AI 中文摘要
本文提出了一种超分辨率（SR）深度学习（DL）架构，用于同时对消费级VL53L9CX飞行时间（ToF）传感器（一种紧凑型单光子雪崩二极管（SPAD）测距设备）获取的低分辨率（LR）深度和强度图像进行上采样，并针对资源受限的微控制器（MCU）部署，该MCU仅具有640 KB SRAM和2 MB Flash。VL53L9CX提供54×42的深度和强度测量。所提出的框架执行×4空间SR，同时重建216×168的深度和强度图像。该网络为强度和高质量（HR）深度采用独立的重建分支。我们使用高效的损失函数训练网络，并通过测试多种紧凑型、面向硬件的骨干网络来研究性能，包括空间自适应特征调制（SAFM）、快速无参数注意力网络（SPAN）和残差局部特征网络（RLFN）。我们在合成和真实测量上评估网络，特别关注受限的激活和内存存储。我们将选定的网络导出到ONNX，量化为INT8，为STM32H563ZI Arm Cortex-M33 MCU生成C推理代码，并将其与简化传感器的固件集成。这项工作首次在低成本裸机MCU上为消费级LR SPAD传感器展示了紧凑型DL SR模型，并与定制MCU固件集成，实现片上SR推理。
英文摘要
This work presents a super-resolution (SR) deep-learning (DL) architecture to simultaneously upscale low-resolution (LR) depth and intensity images acquired by a consumer-grade VL53L9CX time-of-flight (ToF) sensor, a compact single-photon avalanche diode (SPAD) ranging device, while targeting deployment on a resource-constrained microcontroller (MCU) with only 640 KB SRAM and 2 MB Flash. The VL53L9CX provides 54 $\times$ 42 depth and intensity measurements. The proposed framework performs $\times$4 spatial SR to reconstruct 216 $\times$ 168 depth and intensity images simultaneously. The network employs separate reconstruction branches for intensity and high-resolution (HR) depth. We train the network with efficient loss functions and investigate the performance by testing multiple compact, hardware-oriented backbones, including Spatially Adaptive Feature Modulation (SAFM), Swift Parameter-Free Attention Network (SPAN), and Residual Local Feature Network (RLFN). We evaluate the network on both synthetic and real measurements, with particular attention to constrained activation and memory storage. We export the selected network to ONNX, quantize it to INT8, generate C inference code for an STM32H563ZI Arm Cortex-M33 MCU, and integrate it with our simplified sensor's firmware. This work is the first to demonstrate a compact DL SR model on a low-cost bare-metal MCU for a consumer-grade LR SPAD sensor, integrated with customized MCU firmware to enable on-device SR inference.
