This paper addresses the long-standing challenge of reconstructing 3D structures from videos with dynamic content. Current approaches to this problem were not designed to operate on casual videos recorded by standard cameras or require a long optimization time.
Aiming to significantly improve the efficiency of previous approaches, we present TracksTo4D, a learning-based approach that enables inferring 3D structure and camera positions from dynamic content originating from casual videos using a single efficient feed-forward pass. To achieve this, we propose operating directly over 2D point tracks as input and designing an architecture tailored for processing 2D point tracks. Our proposed architecture is designed with two key principles in mind: (1) it takes into account the inherent symmetries present in the input point tracks data, and (2) it assumes that the movement patterns can be effectively represented using a low-rank approximation. TracksTo4D is trained in an unsupervised way on a dataset of casual videos utilizing only the 2D point tracks extracted from the videos, without any 3D supervision. Our experiments show that TracksTo4D can reconstruct a temporal point cloud and camera positions of the underlying video with accuracy comparable to state-of-the-art methods, while drastically reducing runtime by up to 95\%. We further show that TracksTo4D generalizes well to unseen videos of unseen semantic categories at inference time.
RegTrack: Simplicity Beneath Complexity in Robust Multi-Modal 3D Multi-Object Tracking
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（计算机科学与技术学院，南京航空航天大学）；Shenzhen Institute of Research, Nanjing University of Aeronautics and Astronautics（深圳研究院，南京航空航天大学）；Collaborative Innovation Center of Novel Software Technology and Industrialization, Nanjing, China（新型软件技术与产业化协同创新中心，南京，中国）；School of Computer Science and Control Engineering, Shenzhen University of Advanced Technology（计算机科学与控制工程学院，深圳大学先进技术学院）
Re-engineering SORT-based algorithms for low-cost small object tracking from omnidirectional footage
Trinity College Dublin（都柏林圣三一学院）；School of Zoology, Trinity College Dublin（都柏林圣三一学院动物学学院）；Sigmedia Group, Electronic and Electrical Engineering, Trinity College Dublin（都柏林圣三一学院电子与电气工程系Sigmedia团队）
Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video
Inria（法国国家信息与自动化研究所）；Univ Rennes（雷恩大学）；CNRS（法国国家科学研究中心）；IRISA（法国计算机科学与随机系统研究所）；University of Central Florida（中佛罗里达大学）；Google DeepMind（谷歌DeepMind）；University of Amsterdam（阿姆斯特丹大学）；Institute of Advanced Research on Artificial Intelligence (IARAI)（人工智能高级研究所）
PointIT: A Fast Tracking Framework Based on 3D Instance Segmentation
The Hong Kong University of Science and Technology（香港科技大学）
最近最流行的跟踪框架集中在2D图像序列。他们很少跟踪点云中的3D对象。在本文中，我们提出了PointIT，一个快速，简单的跟踪方法的基础上三维道路上的实例分割。首先，我们将3D LiDAR数据转换为尺寸为64 x 512 x 4的球形图像，并将其送入实例片段模型，以获得每个类的预测实例掩码。然后，我们使用MobileNet作为我们的主要编码器，而不是原来的ResNet，以减少计算复杂性。最后，利用该实例框架对排序算法进行了扩展，实现了三维LiDAR点云数据的跟踪。该模型是在球形图像数据集上训练的，具有由KITTI 3D Object Track数据集提供的相应实例标签掩码。实验结果表明，该网络的平均跟踪精度（AP）达到0.617，多目标跟踪性能也得到了提高。
Super Encoding Network: Recursive Association of Multi-Modal Encoders for Video Understanding
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；the School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Shanghai AI Laboratory（上海人工智能实验室）
Task-Aware Encoder Control for Deep Video Compression
Beijing Institute of Technology（北京理工大学）；SenseTime Research（商汤科技研究院）；The Hong Kong University of Science and Technology（香港科技大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Shanghai Jiaotong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）
