Amulet：基于稀疏分层场景表示与自适应着色的帧外推方法
Amulet: Frame Extrapolation Through Sparse Layered Scene Representation and Adaptive Shading
浏览论文内容
中文总结 AI 辅助
Amulet是一种基于稀疏分层图像空间缓存的非神经帧外推渲染方法，可实现高频帧生成，在4K分辨率下最高达250Hz，性能可与DLSS等最先进方法媲美，能准确处理新露出区域且质量高。
中文摘要 AI 辅助
我们提出Amulet，一种将场景转换为稀疏、平铺且分层的中间场景表示（缓存）以实现高频帧外推的渲染方法。与基于重投影的技术不同，Amulet会明确光栅化并存储分层图像空间缓存中可能可见的几何结构，从而能准确对新露出区域进行着色和修复，不会产生幻觉。我们的核心贡献是一种缓存，该缓存会针对未来视图预测性地填充着色信息，并分摊到多个当前帧中。合成新视图时，会通过从前到后分层遍历缓存并动态细化过时或缺失的着色来完成。我们采用基于梯度的预测调度器为每个图块分配生命周期，从而在运动和动态光照下实现自适应着色更新。Amulet将光栅化和着色速率与显示器的刷新率解耦，在许多场景中，其缓存可利用单个着色帧合成多个外推帧，仅需少量局部更新。在典型应用中，我们将60Hz的着色速率外推至240Hz的显示器，Amulet在4K分辨率下最高可达250Hz，且在多项指标上与包括DLSS和神经流方法在内的最先进帧生成方法具有竞争力。Amulet探索了稀疏分层图像空间表示的设计空间，它能实现准确的非神经多帧外推，并明确处理新露出区域。我们的研究结果表明，Amulet可比当代方法外推更多帧且质量更高，在许多场景中，其质量可与受延迟限制的帧插值方法相媲美。
英文摘要
We introduce Amulet, a rendering method that transforms a scene into a sparse, tiled and layered intermediate scene representation (cache) for high-frequency frame extrapolation. In contrast to reprojection-based techniques, Amulet explicitly rasterizes and stores potentially visible geometry in its layered image-space cache, allowing accurate shading and inpainting of newly disoccluded regions without hallucination. Our key contribution is a cache that is predictively filled with shading information for future views, amortized over multiple current frames. Novel views are synthesized by hierarchically traversing the cache front to back and refining stale or missing shading on the fly. Using a predictive, gradient-based scheduler that assigns lifetimes for each tile, we enable adaptive shading updates under motion and dynamic lighting. Amulet decouples the rasterization and shading rate from the refresh rate of the display. In many scenarios, our cache can use a single shaded frame to synthesize multiple extrapolated frames with only a few localized updates. In a typical application, we extrapolate a 60 Hz shading rate to a 240 Hz display. Amulet achieves up to 250 Hz at 4K resolution and is competitive with state-of-the-art frame generation methods, including DLSS and neural-flow approaches, in multiple metrics. Amulet explores the design space of sparse layered image-space representation. It enables accurate, non-neural multi frame extrapolation with explicit handling of disocclusions. Our findings show that Amulet can extrapolate many more frames than contemporary methods with high quality, rivaling latency-bound frame interpolation methods with similar quality in many scenes.
