用于平面视频实时立体转换的轻量级多平面图像网络
Lightweight Multiplane Images Network for Real-Time Stereoscopic Conversion from Planar Video
AI总结：
针对平面视频立体转换中性能与效率难以兼顾的问题，提出含细节分支和轻量深度-语义分支的MPI实时网络，通过训练专用辅助分支和简化渲染，在2K下实现实时推理并较TMPI加速超40倍。
摘要：
随着立体显示技术（尤其是裸眼3D屏幕）和虚拟现实设备的快速发展，立体转换已成为解决高质量立体图像与视频资源匮乏问题的一项重要任务。当前的立体转换算法通常难以在重建性能与推理效率之间取得平衡。本文提出一种基于多平面图像（MPI）的平面视频实时立体转换网络，该网络由用于生成MPI的细节分支和用于感知深度信息的深度-语义分支组成。与依赖显式深度图输入的模型不同，所提方法采用轻量级深度-语义分支来隐式提取深度感知特征。为优化该轻量级分支，采用了“重训练、轻推理”策略，即设计一个仅在训练阶段使用的由粗到细辅助分支。此外，所提方法针对立体转换场景简化了MPI渲染流程，以进一步加快推理速度。实验结果表明，所提方法能够取得与部分最先进（SOTA）模型相当的性能，并支持2K分辨率下的实时推理。与SOTA的TMPI算法相比，所提方法在获得相近主观质量的同时，实现了超过40倍的推理加速。
英文摘要：
With the rapid development of stereoscopic display technologies, especially glasses-free 3D screens, and virtual reality devices, stereoscopic conversion has become an important task to address the lack of high-quality stereoscopic image and video resources. Current stereoscopic conversion algorithms typically struggle to balance reconstruction performance and inference efficiency. This paper proposes a planar video real-time stereoscopic conversion network based on multi-plane images (MPI), which consists of a detail branch for generating MPI and a depth-semantic branch for perceiving depth information. Unlike models that depend on explicit depth map inputs, the proposed method employs a lightweight depth-semantic branch to extract depth-aware features implicitly. To optimize the lightweight branch, a heavy training but light inference strategy is adopted, which involves designing a coarse-to-fine auxiliary branch that is only used during the training stage. In addition, the proposed method simplifies the MPI rendering process for stereoscopic conversion scenarios to further accelerate the inference. Experimental results demonstrate that the proposed method can achieve comparable performance to some state-of-the-art (SOTA) models and support real-time inference at 2K resolution. Compared to the SOTA TMPI algorithm, the proposed method obtains similar subjective quality while achieving over $40\times$ inference acceleration.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
