Deep Analog：基于参考条件3D LUT的开放集胶片模拟
Deep Analog: Open-Set Film Emulation with Reference-Conditioned 3D LUTs
浏览论文内容
中文总结 AI 辅助
本研究提出Deep Analog流水线，通过StyleLUTNet预测参考条件3D LUT实现开放集胶片模拟，结合熵项正则化提升性能，1080p下颜色路径耗时5.2ms，可泛化至未见过的胶片素材。
中文摘要 AI 辅助
胶片模拟是在新的数字照片上重现模拟胶片素材外观的技术，本研究针对其开放集形式——即从单个示例匹配任意参考胶片帧——提出一种由该参考帧预测的3D查找表（LUT）方法。实时图像增强技术会预测固定3D LUT库中各图像的权重并进行混合，该技术本质是一种门控混合专家模型，但存在缺陷：若针对重建进行端到端训练，门会坍缩到单个专家，导致K个LUT库仅能发挥1个的容量。本研究引入熵项（混合专家负载均衡在增强设置中的类似项），可恢复LUT利用率并将峰值信噪比（PSNR）提升约1dB。固定LUT基的更深层限制在于其为闭集，会将可实现的外观冻结在训练时。因此，本研究摒弃该基，提出StyleLUTNet，直接从参考图像预测单个3D LUT作为残差，通过对程序生成的颜色变换进行自监督训练。该条件设计移除了门，无需配对数据或重新训练即可将开放集泛化到未见过的胶片素材。在此颜色主干基础上，本研究构建了Deep Analog胶片模拟流水线，新增基于直方图的色调匹配和物理驱动的光学渲染器——由逆网络从参考帧回归参数驱动的多尺度颗粒和逐通道光晕。在350个自监督对上，颜色阶段达到22.05dB PSNR/0.925结构相似性指数（SSIM），完整流水线达到21.72dB PSNR/0.923；颜色路径在1080p分辨率下运行耗时5.2ms（帧率192FPS），并为标准编辑工具导出可移植的.cube LUT。此外，条件LUT训练中的第二个退化问题——残差尺度坍缩——与上述问题根源相同，由此得到通用原则：辅助正则化必须从属于重建任务。
英文摘要
Film emulation reproduces the look of an analog film stock on a new digital photograph. We target its open-set form -- matching any reference film frame from a single example -- with a 3D lookup table (LUT) predicted from that reference. Real-time image enhancement predicts per-image weights over a fixed bank of 3D LUTs and blends them. We show this is a gated mixture of experts and inherits its failure: trained end-to-end against reconstruction, the gate collapses onto a single expert, so a bank of K LUTs delivers the capacity of one. An entropy term, the enhancement-setting analogue of mixture-of-experts load balancing, restores utilization and recovers about 1 dB PSNR. The deeper constraint survives: a fixed LUT basis is closed-set, freezing the achievable looks at training time. We therefore discard the basis and predict a single 3D LUT as a residual from a reference image (StyleLUTNet), trained by self-supervision on procedurally generated color transforms. The conditional design removes the gate and generalizes open-set to unseen film stocks without paired data or retraining. Around this color backbone we build Deep Analog, a film-emulation pipeline that adds histogram-based tone matching and a physics-informed optical renderer -- multi-scale grain and per-channel halation driven by parameters an inverse network regresses from the reference. On 350 self-supervised pairs the color stage reaches 22.05 dB PSNR / 0.925 SSIM and the full pipeline 21.72 dB / 0.923; the color path runs in 5.2 ms at 1080p (192 FPS) and exports a portable .cube LUT for standard editing tools. A second degeneracy in conditional LUT training -- residual-scale collapse -- shares the root cause and yields a general principle: auxiliary regularization must stay subordinate to reconstruction.
发表机构
- Rochester Institute of Technology（罗切斯特理工学院）
机构由 AI 辅助整理，请以论文原文为准。
