Online augmentation of an oblique aerial image sequence with structural information is an essential aspect in the process of 3D scene interpretation and analysis. One key aspect in this is the efficient dense image matching and depth estimation. Here, the Semi-Global Matching (SGM) approach has proven to be one of the most widely used algorithms for efficient depth estimation, providing a good trade-off between accuracy and computational complexity. However, SGM only models a first-order smoothness assumption, thus favoring fronto-parallel surfaces. In this work, we present a hierarchical algorithm that allows for efficient depth and normal map estimation together with confidence measures for each estimate. Our algorithm relies on a plane-sweep multi-image matching followed by an extended SGM optimization that allows to incorporate local surface orientations, thus achieving more consistent and accurate estimates in areasmade up of slanted surfaces, inherent to oblique aerial imagery. We evaluate numerous configurations of our algorithm on two different datasets using an absolute and relative accuracy measure. In our evaluation, we show that the results of our approach are comparable to the ones achieved by refined Structure-from-Motion (SfM) pipelines, such as COLMAP, which are designed for offline processing. In contrast, however, our approach only considers a confined image bundle of an input sequence, thus allowing to perform an online and incremental computation at 1Hz-2Hz.
FS-I2P:A Hierarchical Focus-Sweep Registration Network with Dynamically Allocated Depth
School of Information Science and Technology, University of Science and Technology of China（信息科学与技术学院，中国科学技术大学）；School of Computer Science and Information Engineering, Hefei University of Technology（计算机科学与信息工程学院，合肥工业大学）；National Key Laboratory of Deep Space Exploration, Deep Space Exploration Laboratory（深空探测国家实验室，深空探测实验室）；Institute of Advanced Technology, University of Science and Technology of China（先进技术研究院，中国科学技术大学）
Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning
Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Department of Computer Science and Engineering, Beihang University（北航计算机科学与工程学院）；Noah’s Ark Lab（诺亚实验室）；Department of Computer Science, University of Hong Kong（香港大学计算机科学系）
This paper presents a novel method, MaskMVS, to solve depth estimation for unstructured multi-view image-pose pairs. In the plane-sweep procedure, the depth planes are sampled by histogram matching that ensures covering the depth range of interest. Unlike other plane-sweep methods, we do not rely on a cost metric to explicitly build the cost volume, but instead infer a multiplane mask representation which regularizes the learning. Compared to many previous approaches, we show that our method is lightweight and generalizes well without requiring excessive training. We outperform the current sta…
Sun Yat-Sen University（中山大学）；University of California, Los Angeles（加州大学洛杉矶分校）
This paper aims at a newly raising task in visual surveillance: re-identifying people at a distance by matching body information, given several reference examples. Most of existing works solve this task by matching a reference template with the target individual, but often suffer from large human appearance variability (e.g. different poses/views, illumination) and high false positives in matching caused by conjunctions, occlusions or surrounding clutters. Addressing these problems, we construct a simple yet expressive template from a few reference images of a certain individual, which represe…
University of Montreal（蒙特利尔大学）；Microsoft Research Lab（微软研究院实验室）
Denoising Score Matching with Annealed Langevin Sampling (DSM-ALS) has recently found success in generative modeling. The approach works by first training a neural network to estimate the score of a distribution, and then using Langevin dynamics to sample from the data distribution assumed by the score network. Despite the convincing visual quality of samples, this method appears to perform worse than Generative Adversarial Networks (GANs) under the Fréchet Inception Distance, a standard metric for generative models. We show that this apparent gap vanishes when denoising the final Langevin sam…
Score-based generative models provide state-of-the-art quality for image and audio synthesis. Sampling from these models is performed iteratively, typically employing a discretized series of noise levels and a predefined scheme. In this note, we first overview three common sampling schemes for models trained with denoising score matching. Next, we focus on one of them, consistent annealed sampling, and study its hyper-parameter boundaries. We then highlight a possible formulation of such hyper-parameter that explicitly considers those boundaries and facilitates tuning when using few or a varia…
Recent end-to-end deep neural networks for disparity regression have achieved the state-of-the-art performance. However, many well-acknowledged specific properties of disparity estimation are omitted in these deep learning algorithms. Especially, matching cost volume, one of the most important procedure, is treated as a normal intermediate feature for the following softargmin regression, lacking explicit constraints compared with those traditional algorithms. In this paper, inspired by previous canonical definition of cost volume, we propose the noise-sampling cross entropy loss function to re…
STORK: Faster Diffusion And Flow Matching Sampling By Resolving Both Stiffness And Structure-Dependence
Department of Mathematics, University of California, Los Angeles（数学系，加州大学洛杉矶分校）；Department of Computer Science, University of California, Los Angeles（计算机科学系，加州大学洛杉矶分校）
In this paper, we focus on the problem of unsupervised image-sentence matching. Existing research explores to utilize document-level structural information to sample positive and negative instances for model training. Although the approach achieves positive results, it introduces a sampling bias and fails to distinguish instances with high semantic similarity. To alleviate the bias, we propose a new sampling strategy to select additional intra-document image-sentence pairs as positive or negative samples. Furthermore, to recognize the complex pattern in intra-document samples, we propose a Tra…
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所机器人学国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；Biorobotics Lab, Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所生物机器人实验室）
Place recognition and loop closure detection are challenging for long-term visual navigation tasks. SeqSLAM is considered to be one of the most successful approaches to achieving long-term localization under varying environmental conditions and changing viewpoints. It depends on a brute-force, time-consuming sequential matching method. We propose MRS-VPR, a multi-resolution, sampling-based place recognition method, which can significantly improve the matching efficiency and accuracy in sequential matching. The novelty of this method lies in the coarse-to-fine searching pipeline and a particle …
Synchronize Feature Extracting and Matching: A Single Branch Framework for 3D Object Tracking
The Hong Kong University of Science and Technology, Guangzhou（香港科技大学（广州））；Zhejiang University（浙江大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；Hangzhou Dianzi University（杭州电子科技大学）
Self-Conditioning Enables Deep Sequential Reasoning in Flow Matching
Shanghai Qi Zhi Institute（上海人工智能实验室）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Georgia Institute of Technology（佐治亚理工学院）；Xiongan AI Institute（雄安人工智能研究院）
LUCID: Learned Undersampling-Adaptive Consistency-Guided Inference with Deterministic Flow Matching for Sparse-View CT Reconstruction
School of Mathematical Sciences, Capital Normal University（首都师范大学数学科学学院）；National Center for Applied Mathematics Beijing, Capital Normal University（首都师范大学北京国家应用数学中心）；Academy for Multidisciplinary Studies, Capital Normal University（首都师范大学交叉科学研究院）
