TrackingWorld: 以世界为中心的单目3D像素跟踪
TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels
AI总结：
TrackingWorld提出了一种以世界为中心的单目3D像素跟踪方法，通过上采样器和优化框架实现对视频中几乎所有像素的密集3D跟踪。
摘要：
单目3D跟踪旨在从单目视频中捕捉3D空间中像素的长期运动，并在近年来取得了快速进展。然而，我们认为现有的单目3D跟踪方法在分离相机运动与前景动态运动方面仍存在不足，且无法在视频中密集跟踪新出现的动态目标。为解决这两个限制，我们提出了TrackingWorld，一种新的管道，用于在以世界为中心的3D坐标系中对几乎所有像素进行密集3D跟踪。首先，我们引入了一个跟踪上采样器，能够高效地将任意稀疏的2D轨迹提升为密集的2D轨迹。然后，为了将当前的跟踪方法推广到新出现的对象，我们将上采样器应用于所有帧，并通过消除重叠区域的轨迹来减少2D轨迹的冗余。最后，我们提出了一种高效的基于优化的框架，通过估计相机姿态和这些2D轨迹的3D坐标，将密集的2D轨迹投影到以世界为中心的3D轨迹中。在合成和真实世界数据集上的广泛评估表明，我们的系统在以世界为中心的坐标框架中实现了准确且密集的3D跟踪。
英文摘要：
Monocular 3D tracking aims to capture the long-term motion of pixels in 3D space from a single monocular video and has witnessed rapid progress in recent years. However, we argue that the existing monocular 3D tracking methods still fall short in separating the camera motion from foreground dynamic motion and cannot densely track newly emerging dynamic subjects in the videos. To address these two limitations, we propose TrackingWorld, a novel pipeline for dense 3D tracking of almost all pixels within a world-centric 3D coordinate system. First, we introduce a tracking upsampler that efficiently lifts the arbitrary sparse 2D tracks into dense 2D tracks. Then, to generalize the current tracking methods to newly emerging objects, we apply the upsampler to all frames and reduce the redundancy of 2D tracks by eliminating the tracks in overlapped regions. Finally, we present an efficient optimization-based framework to back-project dense 2D tracks into world-centric 3D trajectories by estimating the camera poses and the 3D coordinates of these 2D tracks. Extensive evaluations on both synthetic and real-world datasets demonstrate that our system achieves accurate and dense 3D tracking in a world-centric coordinate frame.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
