Efficient-LVSM: 通过解耦共细化注意力实现更高效、更经济、更优质的大型视角合成模型
Efficient-LVSM: Faster, Cheaper, and Better Large View Synthesis Model via Decoupled Co-Refinement Attention
AI总结：
Efficient-LVSM通过解耦共细化注意力机制，实现了更高效、更经济、更优质的大型视角合成模型，提升了PSNR性能并加快了训练和推理速度。
摘要：
针对新型视角合成（NVS）的馈送模型，最近通过基于变压器的方法如LVSM取得了进展，利用输入和目标视角之间的注意力。在本文中，我们指出其完全自注意力设计是次优的，导致输入视角数量的二次复杂度和异质标记之间的刚性参数共享。我们提出了Efficient-LVSM，一种双流架构，通过解耦共细化机制避免了这些问题。它对输入视角应用视内自注意力，对目标视角应用自然后交叉注意力，消除了不必要的计算。Efficient-LVSM在RealEstate10K上以2个输入视角实现了29.86 dB PSNR，超越LVSM 0.2 dB，训练收敛速度加快2倍，推理速度加快4.4倍。Efficient-LVSM在多个基准上实现了最先进的性能，表现出对未见视角数量的强零样本泛化能力，并通过KV-cache实现了增量推理。
英文摘要：
Feedforward models for novel view synthesis (NVS) have recently advanced by transformer-based methods like LVSM, using attention among all input and target views. In this work, we argue that its full self-attention design is suboptimal, suffering from quadratic complexity with respect to the number of input views and rigid parameter sharing among heterogeneous tokens. We propose Efficient-LVSM, a dual-stream architecture that avoids these issues with a decoupled co-refinement mechanism. It applies intra-view self-attention for input views and self-then-cross attention for target views, eliminating unnecessary computation. Efficient-LVSM achieves 29.86 dB PSNR on RealEstate10K with 2 input views, surpassing LVSM by 0.2 dB, with 2x faster training convergence and 4.4x faster inference speed. Efficient-LVSM achieves state-of-the-art performance on multiple benchmarks, exhibits strong zero-shot generalization to unseen view counts, and enables incremental inference with KV-cache, thanks to its decoupled designs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
