事件传感器的最优即时反馈控制
Optimal OnTheFly Feedback Control of Event Sensors
AI总结：
针对事件相机视频重建问题，提出一种数据驱动的即时动态反馈控制方案，通过控制器网络预测最优的每列激活阈值分布，在保证重建质量的同时有效平衡了性能精度与事件率。
摘要：
基于事件的视觉传感器会产生异步事件流，当像素强度变化超过预定义阈值时就会触发这些事件。此类传感器具有显著优势，包括降低数据冗余、微秒级时间分辨率和低功耗，使其在机器人和计算机视觉应用中具有重要价值。在这项工作中，我们考虑了从事件中进行视频重建的问题，并提出了一种激活阈值动态反馈控制方法，其中控制器网络分析过去发出的事件，并预测下一时间段激活阈值的最优分布。此外，我们允许设定用户定义的目标峰值事件率，控制网络以此为条件进行优化，预测每列激活阈值，最终产生最佳的视频重建效果。所提出的即时控制方案是数据驱动的，并使用离散事件表示的概率松弛进行端到端训练。我们证明，在LPIPS感知图像差异度指标上，我们的方法比固定和随机变化阈值方案分别提高了6-12%，在事件率上提高了49%，在实现卓越重建质量的同时，实现了性能精度与事件率之间的精细平衡。此外，我们表明即时控制提供的采样策略是可解释的，并反映了场景的特征。我们的结果源自物理精确的模拟器，突显了所提出方法在增强事件相机用于图像重建和其他下游任务实用性方面的前景，为在硅片中实现动态反馈EVS控制的硬件实现铺平了道路。
英文摘要：
Event-based vision sensors produce an asynchronous stream of events which are triggered when the pixel intensity variation exceeds a predefined threshold. Such sensors offer significant advantages, including reduced data redundancy, micro-second temporal resolution, and low power consumption, making them valuable for applications in robotics and computer vision. In this work, we consider the problem of video reconstruction from events, and propose an approach for dynamic feedback control of activation thresholds, in which a controller network analyzes the past emitted events and predicts the optimal distribution of activation thresholds for the following time segment. Additionally, we allow a user-defined target peak-event-rate for which the control network is conditioned and optimized to predict per-column activation thresholds that would eventually produce the best possible video reconstruction. The proposed OnTheFly control scheme is data-driven and trained in an end-to-end fashion using probabilistic relaxation of the discrete event representation. We demonstrate that our approach outperforms both fixed and randomly-varying threshold schemes by 6-12% in terms of LPIPS perceptual image dissimilarity metric, and by 49% in terms of event rate, achieving superior reconstruction quality while enabling a fine-tuned balance between performance accuracy and the event rate. Additionally, we show that sampling strategies provided by our OnTheFly control are interpretable and reflect the characteristics of the scene. Our results, derived from a physically-accurate simulator, underline the promise of the proposed methodology in enhancing the utility of event cameras for image reconstruction and other downstream tasks, paving the way for hardware implementation of dynamic feedback EVS control in silicon.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
