arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.05388cs.CV

Forest2Seq:复兴顺序先验以用于序列式室内场景合成

Forest2Seq: Revitalizing Order Prior for Sequential Indoor Scene Synthesis

USTC(中国科学技术大学);Simon Fraser University(西蒙菲莎大学)

Qi Sun, Hang Zhou, Wengang Zhou, Li Li, Houqiang Li

AI总结:

该研究提出Forest2Seq框架,通过聚类与广度优先遍历将无序对象组织为有序层级树和森林,并由transformer自回归生成3D室内场景,在FID、KL及下游任务上优于基线。

摘要:

合成逼真的3D室内场景是一项具有挑战性的任务,传统上依赖专业设计师进行手动布置和标注。自回归模型的最新进展已使这一过程自动化,但这些模型往往缺乏对真实场景中存在的关系和层级结构的语义理解,因此性能有限。本文提出Forest2Seq框架,将室内场景合构建模为一个具有顺序感知能力的序列学习问题。Forest2Seq把本质上无序的场景对象集合组织成结构化、有序的层级场景树和森林。通过采用基于聚类的算法和广度优先遍历,Forest2Seq获得有意义的顺序,并利用transformer以自回归方式生成逼真的3D场景。在标准基准上的实验结果表明,与表现最佳的基线方法相比,Forest2Seq在合成更逼真场景方面具有优势,并在FID和KL分数上取得显著提升。我们针对下游任务的额外实验以及消融研究也证实了在3D场景生成中纳入顺序先验的重要性。

英文摘要:

Synthesizing realistic 3D indoor scenes is a challenging task that traditionally relies on manual arrangement and annotation by expert designers. Recent advances in autoregressive models have automated this process, but they often lack semantic understanding of the relationships and hierarchies present in real-world scenes, yielding limited performance. In this paper, we propose Forest2Seq, a framework that formulates indoor scene synthesis as an order-aware sequential learning problem. Forest2Seq organizes the inherently unordered collection of scene objects into structured, ordered hierarchical scene trees and forests. By employing a clustering-based algorithm and a breadth-first traversal, Forest2Seq derives meaningful orderings and utilizes a transformer to generate realistic 3D scenes autoregressively. Experimental results on standard benchmarks demonstrate Forest2Seq's superiority in synthesizing more realistic scenes compared to top-performing baselines, with significant improvements in FID and KL scores. Our additional experiments for downstream tasks and ablation studies also confirm the importance of incorporating order as a prior in 3D scene generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑