Pix2HDR——用于高速 HDR 视频的逐像素采集与基于深度学习的合成方法
Pix2HDR -- A pixel-wise acquisition and deep learning-based synthesis approach for high-speed HDR videos
AI总结：
提出 Pix2HDR 方法，利用逐像素可编程传感器进行变曝光采样，结合深度神经网络合成高时空分辨率的高速 HDR 视频，有效消除运动伪影并解析极端光照下的快速运动。
摘要：
准确捕捉具有大范围运动和光强变化的动态场景对许多视觉应用至关重要。然而，由于相机的帧率限制了其动态范围，获取高速高动态范围（HDR）视频极具挑战。现有方法通过牺牲速度来获取多重曝光帧。但这些帧中的未对齐运动仍会给 HDR 融合算法带来复杂问题，导致伪影。我们不采用基于帧的曝光，而是利用不同曝光和相位偏移下的单个像素对视频进行采样。在单色逐像素可编程图像传感器上实现，我们的采样模式同时以高动态范围捕捉快速运动。随后，我们利用深度神经网络端到端学习的权重将逐像素输出转换为 HDR 视频，实现了高时空分辨率并最小化运动模糊。我们展示了在 1000 FPS 下的无混叠 HDR 视频采集，能够解析低光条件及明亮背景下的快速运动——这两者对传统相机均为挑战。通过结合逐像素采样模式的通用性与深度神经网络解码复杂场景的能力，我们的方法极大提升了视觉系统在动态条件下的适应性和性能。
英文摘要：
Accurately capturing dynamic scenes with wide-ranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera's frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern simultaneously captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds - both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
