MetaSapiens v2：通过注视感知剪枝与立体扭曲推进实时注视感知神经渲染
MetaSapiens v2: Advancing Real-Time Foveated Neural Rendering via Foveation-Aware Pruning and Stereo Warping
浏览论文内容
中文总结 AI 辅助
MetaSapiens v2通过效率感知剪枝、注视感知渲染、选择性扭曲及双目FR加速器四项技术，在保持视觉质量的前提下，将PBNR速度提升一个数量级，解决VR/AR实时渲染挑战。
中文摘要 AI 辅助
基于点的神经渲染（PBNR）正成为一类有前景的渲染技术，受AR/VR与数字孪生领域对实时、 photorealistic渲染需求增长的驱动，正渗透到社会各方面。然而，在VR/AR设备上实现实时PBNR颇具挑战。本文提出MetaSapiens v2，一款可在VR/AR设备上实现实时神经渲染同时保持人类视觉质量的PBNR系统。MetaSapiens v2结合了四项技术：首先，提出效率感知剪枝技术以优化渲染速度；其次，引入适用于PBNR的高效基元的注视感知渲染（FR）方法，利用人类周边区域视觉敏锐度低的特性，放宽渲染质量要求以提升渲染速度；第三，利用双眼间的冗余性，提出选择性扭曲方法，进一步降低AR/VR双目渲染的计算开销；最后，提出双目FR的加速器设计，解决基于FR的PBNR中的负载失衡问题，支持高效双目渲染所需的扭曲操作。我们的评估表明，MetaSapiens v2相较于现有PBNR模型实现了一个数量级的加速，同时保持了视觉质量。
英文摘要
Point-Based Neural Rendering (PBNR) is emerging as a promising class of rendering techniques, which are permeating all aspects of society, driven by a growing demand for real-time, photorealistic rendering in AR/VR and digital twins. However, achieving real-time PBNR on VR/AR devices is challenging. This paper proposes MetaSapiens v2, a PBNR system that delivers real-time neural rendering on VR/AR devices while maintaining human visual quality. MetaSapiens v2 combines four techniques. First, we present an efficiency-aware pruning technique to optimize rendering speed. Second, we introduce a Foveated Rendering (FR) method with an efficient primitive for PBNR, leveraging humans' low visual acuity in peripheral regions to relax rendering quality and improve rendering speed. Third, we leverage the redundancy between the two eyes and propose a selective warping method to further reduce the computation overhead in AR/VR binocular rendering. Finally, we propose an accelerator design for binocular FR, addressing the load imbalance issue in (FR-based) PBNR and supporting warping for efficient binocular rendering. Our evaluation shows that MetaSapiens v2 achieves an order of magnitude speedup over existing PBNR models while maintaining the visual quality.
