arXiv 2503.161342025-03-20cs.CV

用于轻量化 Quad Bayer 混合 EVS 去马赛克的二值化 Mamba-Transformer

Binarized Mamba-Transformer for Lightweight Quad Bayer HybridEVS Demosaicing

Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Harvard University（哈佛大学）；Hong Kong University of Science and Technology, Guangzhou（香港科技大学（广州））

Shiyang Zhou, Haijin Zeng, Yunfan Lu, Tong Shao, Ke Tang, Yongyong Chen, Jie Liu, Jingyong Su

备注：: Accepted by CVPR 2025

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出 BMTNet，通过二值化 Mamba 与 Swin Transformer 混合架构，在保留 Selective Scan 全精度的同时降低 HybridEVS Quad Bayer 去马赛克的计算开销。

摘要：

Quad Bayer 去马赛克是推动混合事件视觉传感器（Hybrid Event-based Vision Sensors，HybridEVS）广泛应用的核心挑战。尽管现有利用长程依赖建模的基于学习方法已取得可观成果，但其复杂性严重限制了在移动设备上的实际部署。为解决这些局限，本文提出一种轻量级的基于 Mamba 的二值神经网络，面向 HybridEVS RAW 图像的高效、高性能去马赛克。首先，为有效捕获全局与局部依赖，我们引入混合的二值化 Mamba-Transformer 架构，结合 Mamba 与 Swin Transformer 架构的优势。其次，为显著降低计算复杂度，我们提出二值化 Mamba（Bi-Mamba），将所有投影二值化，同时以全精度保留核心 Selective Scan。Bi-Mamba 还融入额外的全局视觉信息，以增强全局上下文并缓解精度损失。我们开展定量与定性实验，证明 BMTNet 在性能和计算效率两方面的有效性，提供适合真实边缘设备的轻量级去马赛克方案。代码和模型位于 https://github.com/Clausy9/BMTNet。

英文摘要：

Quad Bayer demosaicing is the central challenge for enabling the widespread application of Hybrid Event-based Vision Sensors (HybridEVS). Although existing learning-based methods that leverage long-range dependency modeling have achieved promising results, their complexity severely limits deployment on mobile devices for real-world applications. To address these limitations, we propose a lightweight Mamba-based binary neural network designed for efficient and high-performing demosaicing of HybridEVS RAW images. First, to effectively capture both global and local dependencies, we introduce a hybrid Binarized Mamba-Transformer architecture that combines the strengths of the Mamba and Swin Transformer architectures. Next, to significantly reduce computational complexity, we propose a binarized Mamba (Bi-Mamba), which binarizes all projections while retaining the core Selective Scan in full precision. Bi-Mamba also incorporates additional global visual information to enhance global context and mitigate precision loss. We conduct quantitative and qualitative experiments to demonstrate the effectiveness of BMTNet in both performance and computational efficiency, providing a lightweight demosaicing solution suited for real-world edge devices. Our codes and models are available at https://github.com/Clausy9/BMTNet.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。