Feed-forward multi-frame 3D reconstruction models often degrade on videos with object motion. Global-reference becomes ambiguous under multiple motions, while the local pointmap relies heavily on estimated relative poses and can drift, causing cross-frame misalignment and duplicated structures. We propose TrajVG, a reconstruction framework that makes cross-frame 3D correspondence an explicit prediction by estimating camera-coordinate 3D trajectories. We couple sparse trajectories, per-frame local point maps, and relative camera poses with geometric consistency objectives: (i) bidirectional trajectory-pointmap consistency with controlled gradient flow, and (ii) a pose consistency objective driven by static track anchors that suppresses gradients from dynamic regions. To scale training to in-the-wild videos where 3D trajectory labels are scarce, we reformulate the same coupling constraints into self-supervised objectives using only pseudo 2D tracks, enabling unified training with mixed supervision. Extensive experiments across 3D tracking, pose estimation, pointmap reconstruction, and video depth show that TrajVG surpasses the current feedforward performance baseline.
SegmentDreamer: Towards High-fidelity Text-to-3D Synthesis with Segmented Consistency Trajectory Distillation
Sun Yat-sen University（中山大学）；Great Bay University（大贝大学）；Pazhou Lab (Huangpu)（琶洲实验室）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）
Huazhong University of Science and Technology（华中科技大学）
Visual images usually contain the informative context of the environment, thereby helping to predict agents' behaviors. However, they hardly impose the dynamic effects on agents' actual behaviors due to the respectively fixed semantics. To solve this problem, we propose a deterministic model named BGM to construct a guidance map to represent the dynamic semantics, which circumvents to use visual images for each agent to reflect the difference of activities in different periods. We first record all agents' activities in the scene within a period close to the current to construct a guidance map …
End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting
Beijing Institute of Technology（北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Shenzhen MSU-BIT University（深圳北理莫斯科大学）；University of Rochester（罗切斯特大学）
Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning
University of Macau（澳门大学）；University of Macau Advanced Research Institute in Hengqin（澳门大学横琴研究院）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Shanghai Jiao Tong University（上海交通大学）；Tongren Hospital（同仁医院）；Duke University（杜克大学）
VisualErase: Dual-Branch Visual Trajectory Redirection for Robust Concept Erasure in Text-to-Image Diffusion Models
Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；City University of Hong Kong（香港城市大学）；Shenzhen Loop Area Institute（深圳河套学院）；National University of Singapore（新加坡国立大学）；Shandong University（山东大学）
TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos
University of Electronic Science and Technology of China（电子科学与技术大学）；Peking University（北京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Zhongguancun Academy（中关村学院）
We present a reward-predictive, model-based deep learning method featuring trajectory-constrained visual attention for local planning in visual navigation tasks. Our method learns to place visual attention at locations in latent image space which follow trajectories caused by vehicle control actions to enhance predictive accuracy during planning. The attention model is jointly optimized by the task-specific loss and an additional trajectory-constraint loss, allowing adaptability yet encouraging a regularized structure for improved generalization and reliability. Importantly, visual attention i…
We present an unsupervised deep neural network approach to the fusion of RGB-D imagery with inertial measurements for absolute trajectory estimation. Our network, dubbed the Visual-Inertial-Odometry Learner (VIOLearner), learns to perform visual-inertial odometry (VIO) without inertial measurement unit (IMU) intrinsic parameters (corresponding to gyroscope and accelerometer bias or white noise) or the extrinsic calibration between an IMU and camera. The network learns to integrate IMU measurements and generate hypothesis trajectories which are then corrected online according to the Jacobians o…
Automated Parking is becoming a standard feature in modern vehicles. Existing parking systems build a local map to be able to plan for maneuvering towards a detected slot. Next generation parking systems have an use case where they build a persistent map of the environment where the car is frequently parked, say for example, home parking or office parking. The pre-built map helps in re-localizing the vehicle better when its trying to park the next time. This is achieved by augmenting the parking system with a Visual SLAM pipeline and the feature is called trained trajectory parking in the auto…
TrajFusionNet: Pedestrian Crossing Intention Prediction via Fusion of Sequential and Visual Trajectory Representations
Perception, Robotics and Intelligent Machines Research Group (PRIME) Department of Computer Science, Université de Moncton, Canada（感知、机器人与智能机器研究组（PRIME）计算机科学系，新不伦瑞克大学）
Stanford University（斯坦福大学）；Tencent AI Lab（腾讯人工智能实验室）
Perceiving and manipulating 3D articulated objects (e.g., cabinets, doors) in human environments is an important yet challenging task for future home-assistant robots. The space of 3D articulated objects is exceptionally rich in their myriad semantic categories, diverse shape geometry, and complicated part functionality. Previous works mostly abstract kinematic structure with estimated joint parameters and part poses as the visual representations for manipulating 3D articulated objects. In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshak…
Technical University of Munich（慕尼黑工业大学）；Artisense（阿蒂森斯公司）
Vision-based learning methods for self-driving cars have primarily used supervised approaches that require a large number of labels for training. However, those labels are usually difficult and expensive to obtain. In this paper, we demonstrate how a model can be trained to control a vehicle's trajectory using camera poses estimated through visual odometry methods in an entirely self-supervised fashion. We propose a scalable framework that leverages trajectory information from several different runs using a camera setup placed at the front of a car. Experimental results on the CARLA simulator …
This paper proposes a novel framework to alleviate the model drift problem in visual tracking, which is based on paced updates and trajectory selection. Given a base tracker, an ensemble of trackers is generated, in which each tracker's update behavior will be paced and then traces the target object forward and backward to generate a pair of trajectories in an interval. Then, we implicitly perform self-examination based on trajectory pair of each tracker and select the most robust tracker. The proposed framework can effectively leverage temporal context of sequential frames and avoid to learn …
VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generation
Shanghai University Shanghai Engineering Research Center of Motion Picture Special Effects LIX, \'Ecole Polytechnique, CNRS, IPP magenta http://vertigo.magic-lab.tech/
Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characters
Institute of Information Engineering, Chinese Academy of Sciences（信息工程研究所，中国科学院）；Tencent（腾讯）；Anyang Normal University（安阳师范学院）；The Palace Museum（故宫博物院）；Nankai University（南开大学）
UltraDub: Towards Authentic Dubbing by Unifying Visually-Steered Flow Learning and Trajectory Guidance
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Hangzhou Dianzi University（杭州电子科技大学）；ByteDance（字节跳动）；University of Science and Technology of China（中国科学技术大学）
SCT-MOT: Enhancing Air-to-Air Multiple UAVs Tracking with Swarm-Coupled Motion and Trajectory Guidance
China-UAE Belt and Road Joint Laboratory on Intelligent Unmanned Systems of School of Aerospace Engineering, Beijing Institute of Technology（北京理工大学宇航学院中国-阿联酋‘一带一路’智能无人系统联合实验室）；School of Aerospace Engineering, Beijing Institute of Technology（北京理工大学宇航学院）；School of Information and Computing Science, Xi'an Jiaotong-Liverpool University（西交利物浦大学信息与计算科学学院）
Video relation detection problem refers to the detection of the relationship between different objects in videos, such as spatial relationship and action relationship. In this paper, we present video relation detection with trajectory-aware multi-modal features to solve this task. Considering the complexity of doing visual relation detection in videos, we decompose this task into three sub-tasks: object detection, trajectory proposal and relation prediction. We use the state-of-the-art object detection method to ensure the accuracy of object trajectory detection and multi-modal feature represe…
Free-VSC: Free Semantics from Visual Foundation Models for Unsupervised Video Semantic Compression
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
TrajFusionNet+: Transformer-Based Prediction of Pedestrian Crossing Intention via Fusion of Trajectory Representations and Scene Graphs
Perception, Robotics and Intelligent Machines Research Group (PRIME)（感知、机器人与智能机器研究组（PRIME））；Department of Computer Science, Université de Moncton（蒙克顿大学计算机科学系）
Wuhan University of Technology（武汉理工大学）；National Engineering Research Center for Water Transport Safety（国家水运安全工程技术研究中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
The automatic identification system (AIS) and video cameras have been widely exploited for vessel traffic surveillance in inland waterways. The AIS data could provide the vessel identity and dynamic information on vessel position and movements. In contrast, the video data could describe the visual appearances of moving vessels, but without knowing the information on identity, position and movements, etc. To further improve vessel traffic surveillance, it becomes necessary to fuse the AIS and video data to simultaneously capture the visual features, identity and dynamic information for the vess…
AV-DTEC: Self-Supervised Audio-Visual Fusion for Drone Trajectory Estimation and Classification
College of Automation Engineering, Nanhang University（南京航空航天大学自动化工程学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
Institute of Mathematics of the Romanian Academy（罗马尼亚科学院数学研究所）；Faculty of Automatic Control and Computers, University Politehnica of Bucharest（布加勒斯特理工大学自动控制与计算机学院）
When driving, people make decisions based on current traffic as well as their desired route. They have a mental map of known routes and are often able to navigate without needing directions. Current self-driving models improve their performances when using additional GPS information. Here we aim to push forward self-driving research and perform route planning even in the absence of GPS. Our system learns to predict in real-time vehicle's current location and future trajectory, as a function of time, on a known map, given only the raw video stream and the intended destination. The GPS signal is…
This paper presents a method to reconstruct dense semantic trajectory stream of human interactions in 3D from synchronized multiple videos. The interactions inherently introduce self-occlusion and illumination/appearance/shape changes, resulting in highly fragmented trajectory reconstruction with noisy and coarse semantic labels. Our conjecture is that among many views, there exists a set of views that can confidently recognize the visual semantic label of a 3D trajectory. We introduce a new representation called 3D semantic map---a probability distribution over the semantic labels per traject…
WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving
The Hong Kong University of Science and Technology（香港科技大学）；Horizon Robotics（地平线机器人）；The Chinese University of Hong Kong（香港中文大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Nankai University（南开大学）
MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidance
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University（上海智能信息处理关键实验室，复旦大学计算机学院）；Shanghai Collaborative Innovation Center of Intelligent Visual Computing（上海智能视觉计算协同创新中心）；Microsoft Research Asia（微软亚洲研究院）
the Hong Kong University of Science and Technology（香港科技大学）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Pavement condition is crucial for civil infrastructure maintenance. This task usually requires efficient road damage localization, which can be accomplished by the visual odometry system embedded in unmanned aerial vehicles (UAVs). However, the state-of-the-art visual odometry and mapping methods suffer from large drift under the degeneration of the scene structure. To alleviate this issue, we integrate normal constraints into the visual odometry process, which greatly helps to avoid large drift. By parameterizing the normal vector on the tangential plane, the normal factors are coupled with t…
