从散焦到聚焦：通过融合散焦与辐射度先验实现照片级真实的散景渲染
Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priors
AI总结：
本文提出 D2F 框架，利用散焦幻觉、辐射度虚拟化与分层融合，从单张全聚焦图像生成物理准确且保持清晰聚焦区域的真实散景，并在挑战赛中获亚军。
摘要：
我们研究从单张全聚焦图像进行真实散景渲染的问题。散景渲染模拟专业摄影中富有美感的浅景深（DoF），但现有方法生成的这些视觉效果存在背景模糊简单扁平以及聚焦区域被模糊的问题，导致渲染结果不真实。在本文中，我们认为真实的散景渲染应当：（i）建模深度关系并区分聚焦区域，（ii）保持聚焦区域清晰，以及（iii）渲染物理准确的弥散圆（CoC）。为此，我们提出了 Defocus to Focus（D2F）框架，通过将散焦先验与全聚焦图像融合，并在分层融合中引入辐射度先验，来学习真实的散景渲染。由于没有提供深度图，我们引入散焦幻觉，通过学习聚焦来整合深度。预测得到的散焦图隐含散景的模糊量，并用于引导加权分层渲染。在分层渲染中，我们根据散焦图融合由不同核模糊的图像。为增强散景的真实感，我们采用辐射度虚拟化来模拟场景辐射度。加权分层渲染中使用的场景辐射度会重新分配软圆盘核中的权重，以生成 CoC。为确保聚焦区域的清晰度，我们建议融合上采样散景图像与原始图像。我们根据散焦图预测初始融合掩膜，并利用深度网络对该掩膜进行细化。我们在一个大规模散景数据集上评估了模型。大量实验表明，我们的方法能够在复杂场景中渲染视觉上令人愉悦的散景效果。特别地，我们的方案在 AIM 2020 Rendering Realistic Bokeh Challenge 中获得亚军。
英文摘要：
We consider the problem of realistic bokeh rendering from a single all-in-focus image. Bokeh rendering mimics aesthetic shallow depth-of-field (DoF) in professional photography, but these visual effects generated by existing methods suffer from simple flat background blur and blurred in-focus regions, giving rise to unrealistic rendered results. In this work, we argue that realistic bokeh rendering should (i) model depth relations and distinguish in-focus regions, (ii) sustain sharp in-focus regions, and (iii) render physically accurate Circle of Confusion (CoC). To this end, we present a Defocus to Focus (D2F) framework to learn realistic bokeh rendering by fusing defocus priors with the all-in-focus image and by implementing radiance priors in layered fusion. Since no depth map is provided, we introduce defocus hallucination to integrate depth by learning to focus. The predicted defocus map implies the blur amount of bokeh and is used to guide weighted layered rendering. In layered rendering, we fuse images blurred by different kernels based on the defocus map. To increase the reality of the bokeh, we adopt radiance virtualization to simulate scene radiance. The scene radiance used in weighted layered rendering reassigns weights in the soft disk kernel to produce the CoC. To ensure the sharpness of in-focus regions, we propose to fuse upsampled bokeh images and original images. We predict the initial fusion mask from our defocus map and refine the mask with a deep network. We evaluate our model on a large-scale bokeh dataset. Extensive experiments show that our approach is capable of rendering visually pleasing bokeh effects in complex scenes. In particular, our solution receives the runner-up award in the AIM 2020 Rendering Realistic Bokeh Challenge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
