DeepLight:用于真实世界显示的鲁棒且无干扰的实时的屏幕-摄像机通信
DeepLight: Robust & Unobtrusive Real-time Screen-Camera Communication for Real-World Displays
摘要:
本文介绍了一种新的,整体的方法,用于强大的屏幕摄像机通信(SCC),其中屏幕上的视频内容在视觉上编码在人类无法感知的方式和解码的摄像机捕捉图像的屏幕内容。我们首先表明,最先进的SCC技术有两个关键的限制,在野外部署:(a)解码准确性迅速下降,即使适度的屏幕提取错误从捕获的图像,和(b)他们产生可感知的闪烁常见的刷新率屏幕上,即使最小的像素强度调制。为了克服这些挑战,我们引入了DeepLight,这是一种在解码管道中集成了机器学习(ML)模型的系统,可在各种现实条件下实现人类无法感知的中高SCC率。Deep-Light的关键创新是设计了一种基于深度神经网络(DNN)的解码器,该解码器可以对显示帧中空间编码的所有比特进行集体解码,而无需尝试精确隔离与每个编码比特相关的像素。此外,DeepLight通过选择性地仅调制蓝色通道的强度来支持不可感知的编码,并通过使用最先进的目标检测DNN管道来提供合理准确的屏幕提取(IoU值>= 83%)。我们表明,一个功能齐全的DeepLight系统能够鲁棒地实现高解码精度(帧出错率< 0.2)和适度高的数据吞吐量(>= 0.95 Kbps)使用手持智能手机相机,即使在更大的屏幕相机的距离(约= 2米)。
英文摘要:
The paper introduces a novel, holistic approach for robust Screen-Camera Communication (SCC), where video content on a screen is visually encoded in a human-imperceptible fashion and decoded by a camera capturing images of such screen content. We first show that state-of-the-art SCC techniques have two key limitations for in-the-wild deployment: (a) the decoding accuracy drops rapidly under even modest screen extraction errors from the captured images, and (b) they generate perceptible flickers on common refresh rate screens even with minimal modulation of pixel intensity. To overcome these challenges, we introduce DeepLight, a system that incorporates machine learning (ML) models in the decoding pipeline to achieve humanly-imperceptible, moderately high SCC rates under diverse real-world conditions. Deep-Light's key innovation is the design of a Deep Neural Network (DNN) based decoder that collectively decodes all the bits spatially encoded in a display frame, without attempting to precisely isolate the pixels associated with each encoded bit. In addition, DeepLight supports imperceptible encoding by selectively modulating the intensity of only the Blue channel, and provides reasonably accurate screen extraction (IoU values >= 83%) by using state-of-the-art object detection DNN pipelines. We show that a fully functional DeepLight system is able to robustly achieve high decoding accuracy (frame error rate < 0.2) and moderately-high data goodput (>=0.95Kbps) using a human-held smartphone camera, even over larger screen-camera distances (approx =2m).
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。