UMIFormer：挖掘相似 token 之间的相关性以用于多视图 3D 重建
UMIFormer: Mining the Correlations between Similar Tokens for Multi-View 3D Reconstruction
AI总结：
针对非结构化多视图缺乏先验关联的问题，本文提出 UMIFormer，通过视图内编码、相似 token 校正及视图间信息聚合实现多视图 3D 重建，并在 ShapeNet 上显著超越 SOTA。
摘要：
近年来，许多视频任务通过利用 vision transformer 并建立用于特征提取的时空解耦而取得了突破。尽管多视图 3D 重建同样面对多张图像作为输入，但由于非结构化视图之间的关联完全模糊，它无法直接继承这些成功。这里不存在可用的先验关系，类似于视频中的时间相干属性。为解决这一问题，我们提出了一种用于非结构化多图像的新型 transformer 网络 UMIFormer。它利用 transformer 块进行解耦的视图内编码，并设计了用于 token 校正的块，通过挖掘来自不同视图的相似 token 之间的相关性来实现解耦的视图间编码。随后，利用 token 之间的相似性，将从各个分支获得的所有 token 压缩为固定大小的紧凑表示，同时为重建保留丰富信息。我们在 ShapeNet 上进行了实证证明，并确认我们的解耦学习方法适用于非结构化多图像。同时，实验还验证了我们的模型以较大优势优于现有的 SOTA 方法。代码将在 https://github.com/GaryZhu1996/UMIFormer 提供。
英文摘要：
In recent years, many video tasks have achieved breakthroughs by utilizing the vision transformer and establishing spatial-temporal decoupling for feature extraction. Although multi-view 3D reconstruction also faces multiple images as input, it cannot immediately inherit their success due to completely ambiguous associations between unstructured views. There is not usable prior relationship, which is similar to the temporally-coherence property in a video. To solve this problem, we propose a novel transformer network for Unstructured Multiple Images (UMIFormer). It exploits transformer blocks for decoupled intra-view encoding and designed blocks for token rectification that mine the correlation between similar tokens from different views to achieve decoupled inter-view encoding. Afterward, all tokens acquired from various branches are compressed into a fixed-size compact representation while preserving rich information for reconstruction by leveraging the similarities between tokens. We empirically demonstrate on ShapeNet and confirm that our decoupled learning method is adaptable for unstructured multiple images. Meanwhile, the experiments also verify our model outperforms existing SOTA methods by a large margin. Code will be available at https://github.com/GaryZhu1996/UMIFormer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
