发表机构
Macau University of Science and Technology; The University of Queensland; Great Bay University; Tsinghua University; National University of Singapore; Institute of Automation, Chinese Academy of Sciences (CASIA)（澳门科技大学; 昆士兰大学; 大湾区大学; 清华大学; 新加坡国立大学; 中国科学院自动化研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出Uni4R框架，通过最优传输与常微分方程协同学习连续速度场，结合流匹配引导解码器与积分一致性训练策略，实现统一连续时间4D重建与点跟踪，在相关任务及新基准上达SOTA性能。
AI 中文摘要
现有统一4D重建与点跟踪方法通常依赖启发式插值或仅在整数时间戳进行预测，缺乏运动学一致性，无法建模任意时间戳的动态。本文提出Uni4R框架，通过最优传输（OT）与常微分方程（ODE）协同学习连续速度场，以统一上述两项任务。该连续速度场作为运动学先验，同时惠及4D重建与点跟踪。具体而言，本文提出流匹配引导解码器（FMGD）：全局速度分支先提取锚点特征，捕捉序列的全局动态状态；FMGD利用流匹配（FM）理论在锚点特征流形上构建由OT定义的概率路径，将其实例化为FM引导的速度特征用于速度预测，从而建立稳健的运动学归纳偏置。同时，点重建分支提供几何特征，局部速度预测模块将上述特征与时间嵌入相结合，解码任意时间戳的速度。为克服分数帧中缺乏高质量真实速度的问题，本文提出积分一致性训练策略：该策略使用ODE求解器积分速度以恢复目标点云，使模型可直接从整数时间戳进行端到端监督。实验结果表明，Uni4R在4D重建与点跟踪任务中均达到SOTA性能，且在本文提出的连续时间运动学感知基准上也达到SOTA。
英文摘要
Existing unified 4D reconstruction and point tracking approaches typically rely on heuristic interpolations or just predict at integer timestamps, lacking kinematic coherence and failing to model dynamics at any arbitrary timestamp. In this paper, we propose Uni4R, a framework that unifies these tasks by learning continuous velocity fields through the synergy of Optimal Transport (OT) and Ordinary Differential Equation (ODE). Importantly, this continuous velocity field acts as a kinematic prior that mutually benefits both 4D reconstruction and point tracking. Specifically, we propose the Flow Matching Guided Decoder (FMGD). A global velocity branch first extracts anchor features that capture the global dynamic state of the sequence. Then, FMGD leverages Flow Matching (FM) theory to formulate a probability path defined by OT on the anchor feature manifold, instantiating it as FM-guided velocity features for velocity prediction. This establishes a robust kinematic inductive bias. Meanwhile, a point reconstruction branch provides geometric features. The local velocity prediction module then joint above features and time embeddings, to decode velocities at arbitrary timestamps. To overcome the absence of high-quality ground-truth velocities in fractional frames, we propose an integral-consistency training strategy. This strategy uses an ODE solver to integrate velocities to recover target pointmaps, enabling the model to be supervised end-to-end directly from integer timestamps. Experimental results demonstrate that Uni4R achieves SOTA performance in both 4D reconstruction and point tracking, and achieves SOTA in our new kinematics-aware benchmark at continuous time.
CommentsPreliminary version
