arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.11917cs.CV
收藏

PicoSAM3:实时传感器区域感兴趣分割

PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

ETH Zürich(苏黎世联邦理工学院);IBM Research(IBM研究院)

Pietro Bonazzi, Nicola Farronato, Stefan Zihlmann, Haotong Qin, Michele Magno

AI总结:

PicoSAM3是一款轻量级实时传感器区域分割模型,结合密集CNN架构、区域兴趣提示编码和知识蒸馏,实现低延迟高精度分割。

摘要:

实时、在设备上的分割对于延迟敏感且隐私保护的应用至关重要,如智能眼镜和物联网设备。我们介绍了PicoSAM3,一个针对边缘和传感器执行优化的轻量级提示视觉分割模型,包括在索尼IMX500视觉传感器上的部署。PicoSAM3拥有1.3M参数,结合密集CNN架构、区域兴趣提示编码、高效通道注意机制以及从SAM2和SAM3的知识蒸馏。在COCO和LVIS数据集上,PicoSAM3分别达到65.45%和64.01%的mIoU,优于现有基于SAM和边缘导向的基线模型。INT8量化模型在精度上几乎没有下降,同时在IMX500上实现了11.82ms的实时传感器推断延迟,完全符合其内存和运算限制。消融研究显示,从大SAM模型的知识蒸馏可使mIoU提升高达14.5%,证明了高质量、空间灵活的提示分割可在传感器层面实现。

英文摘要:

Real-time, on-device segmentation is critical for latency-sensitive and privacy-aware applications such as smart glasses and Internet-of-Things devices. We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution, including deployment on the Sony IMX500 vision sensor. PicoSAM3 has 1.3M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines at similar or lower complexity. The INT8 quantized model preserves accuracy with negligible degradation while enabling real-time in-sensor inference at 11.82ms latency on the IMX500, fully complying with its memory and operator constraints. Ablation studies show that distillation from large SAM models yields up to +14.5% mIoU improvement over supervised training and demonstrate that high-quality, spatially flexible promptable segmentation is feasible directly at the sensor level.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑