Temporal motion modeling has always been a key component in multiple object tracking (MOT) which can ensure smooth trajectory movement and provide accurate positional information to enhance association precision. However, current motion models struggle to be both efficient and effective across different application scenarios. To this end, we propose TrackSSM inspired by the recently popular state space models (SSM), a unified encoder-decoder motion framework that uses data-dependent state space model to perform temporal motion of trajectories. Specifically, we propose Flow-SSM, a module that utilizes the position and motion information from historical trajectories to guide the temporal state transition of object bounding boxes. Based on Flow-SSM, we design a flow decoder. It is composed of a cascaded motion decoding module employing Flow-SSM, which can use the encoded flow information to complete the temporal position prediction of trajectories. Additionally, we propose a Step-by-Step Linear (S$^2$L) training strategy. By performing linear interpolation between the positions of the object in the previous frame and the current frame, we construct the pseudo labels of step-by-step linear training, ensuring that the trajectory flow information can better guide the object bounding box in completing temporal transitions. TrackSSM utilizes a simple Mamba-Block to build a motion encoder for historical trajectories, forming a temporal motion model with an encoder-decoder structure in conjunction with the flow decoder. TrackSSM is applicable to various tracking scenarios and achieves excellent tracking performance across multiple benchmarks, further extending the potential of SSM-like temporal motion models in multi-object tracking tasks. Code and models are publicly available at \url{https://github.com/Xavier-Lin/TrackSSM}.
MotionTrack: Learning Motion Predictor for Multiple Object Tracking
School of Computer Science, National University of Defense Technology(国防科技大学计算机学院);JD Explore Academy(京东探索研究院);Meituan Inc.(美团公司);Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology(国防科技大学前沿装备数字化软件实验室)
Time-series Meets Complex Motion Modeling: Robust and Computational-effective Motion Predictor for Multi-object Tracking
Faculty of Information Science and Engineering, University of Information Technology, Ho Chi Minh City, Vietnam(1* 信息科学与工程学院,信息技术大学,胡志明市,越南);Vietnam National University, Ho Chi Minh City, Vietnam(2 越南国家大学,胡志明市,越南)
Masked Motion Predictors are Strong 3D Action Representation Learners
University of Science and Technology of China(中国科学技术大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);The University of Sydney(悉尼大学);Merchants Union Consumer Finance Company Limited(招商局消费金融有限公司)
P4P: Conflict-Aware Motion Prediction for Planning in Autonomous Driving
Tsinghua University(清华大学);Shanghai Qi Zhi Institute(上海期智研究院);Massachusetts Institute of Technology(麻省理工学院)
运动预测对于在交互式场景中实现自动驾驶车辆的安全运动规划至关重要。它允许规划者识别与其他交通代理的潜在冲突并生成安全的计划。现有的运动预测器通常专注于减少预测误差,但它们如何帮助规划器识别冲突仍然是一个悬而未决的问题。在本文中,我们通过新的冲突相关的指标,如识别冲突的成功率,评估国家的最先进的预测。令人惊讶的是,预测遭受低成功率,从而导致很大比例的碰撞时,我们在一个交互式模拟器中测试的预测规划系统。为了填补这一空白,我们提出了一个简单但有效的替代方案,它结合了基于物理的轨迹生成器和基于学习的关系预测器来识别冲突并推断冲突关系。我们证明了我们的预测器P4P在Waymo Open Motion Dataset的现实交互式驾驶场景中比现有的基于学习的预测器具有更好的性能。
Shanghai Jiao Tong University(上海交通大学);University of California, Berkeley(加州大学伯克利分校)
基于点云的三维场景流估计是计算机视觉中的一个底层三维运动感知任务。流嵌入是场景流估计中常用的一种技术,它对相邻帧之间的点运动进行编码。因此,流嵌入捕捉正确的运动总体方向是至关重要的。然而,以前的作品只搜索本地,以确定一个软对应,忽略了遥远的点,原来是实际的匹配。此外,估计的对应关系通常来自相邻点云的前向方向,并且可能与从后向方向获取的估计的对应关系不一致。为了解决这些问题,我们提出了一种新的全对全流嵌入层,在初始场景流估计过程中进行向后可靠性验证。此外,我们还研究和比较了三维场景流网络关键组件的几种设计选择,包括点相似度计算、预测器输入元素、预测器和细化级设计。在仔细选择了最有效的设计之后,我们能够呈现一个在FlyingThings3D和KITTI场景流数据集上实现最先进性能的模型。我们提出的模型在FlyingThings3D数据集上超过所有现有方法至少38.2%,在KITTI Scene Flow数据集上超过24.7%。我们在https://github.com/IRMVLab/3DFlow上发布代码。
Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video
The Chinese University of Hong Kong(香港中文大学);Shun Hing Institute of Advanced Engineering, CUHK(香港中文大学晨兴先进工程研究所);T Stone Robotics Institute, CUHK(香港中文大学T Stone机器人研究所)
College of Artificial Intelligence and Automation, Hohai University(人工智能与自动化学院,河海大学);College of DaYu, Hohai University(大禹学院,河海大学);College of Water Conserwancy and Hydropower Engineering, Hohai University(水利水电工程学院,河海大学);College of Computer Science and Technology, Zhejiang University of Technology(计算机科学与技术学院,浙江工业大学)
TaPD: Temporal-adaptive Progressive Distillation for Observation-Adaptive Trajectory Forecasting in Autonomous Driving
College of Information and Intelligent Science, Donghua University(信息与智能科学学院,东华大学);School of Computing and Information Technology, Great Bay Institute for Advanced Study/Great Bay University(计算与信息技术学院,大澳研究院/大澳大学);Chongqing Chang’an Technology Co., Ltd(重庆长安科技有限公司);ViSiR, Reutlingen University(ViSiR,鲁特大学)