Camera pose regression methods apply a single forward pass to the query image to estimate the camera pose. As such, they offer a fast and light-weight alternative to traditional localization schemes based on image retrieval. Pose regression approaches simultaneously learn two regression tasks, aiming to jointly estimate the camera position and orientation using a single embedding vector computed by a convolutional backbone. We propose an attention-based approach for pose regression, where the convolutional activation maps are used as sequential inputs. Transformers are applied to encode the sequential activation maps as latent vectors, used for camera pose regression. This allows us to pay attention to spatially-varying deep features. Using two Transformer heads, we separately focus on the features for camera position and orientation, based on how informative they are per task. Our proposed approach is shown to compare favorably to contemporary pose regressors schemes and achieves state-of-the-art accuracy across multiple outdoor and indoor benchmarks. In particular, to the best of our knowledge, our approach is the only method to attain sub-meter average accuracy across outdoor scenes. We make our code publicly available from here.
XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface
Department of Translational Surgical Oncology, National Center for Tumor Diseases (NCT), NCT/UCC Dresden, a partnership between DKFZ, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, and Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Germany（转化外科肿瘤学系，国家肿瘤疾病中心（NCT），NCT/UCC德累斯顿，由德累斯顿癌症研究中心（DKFZ）、医学院和德累斯顿卡尔·戈斯特·卡尔医院、德累斯顿技术大学（TUD）以及德累斯顿-罗斯托克中心（HZDR）组成的联合体，德国）；Department of Visceral, Thoracic and Vascular Surgery, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD, Germany（visceral、胸腔和血管外科系，医学院和卡尔·戈斯特·卡尔医院，TUD，德国）；Cluster of Excellence-CeTI, TUD, Germany（卓越中心-CeTI，TUD，德国）
Microsoft Corp.（微软公司）；University of Campinas（坎皮纳斯大学）
Person Re-Identification is a challenging task that aims to retrieve all instances of a query image across a system of non-overlapping cameras. Due to the various extreme changes of view, it is common that local regions that could be used to match people are suppressed, which leads to a scenario where approaches have to evaluate the similarity of images based on less informative regions. In this work, we introduce the Top-DB-Net, a method based on Top DropBlock that pushes the network to learn to focus on the scene foreground, with special emphasis on the most task-relevant regions and, at the…
Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. However, class-insensitivity of the earlier layers in a network only allows saliency computation with low resolution activation maps of the deeper layers, resulting in compromised image saliency. Remedifying this can lead to sanity failures. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors and preserving the map sanity. Our method systematically performs multi-scale accumulation …
Deep neural networks (DNNs) have a high accuracy on image classification tasks. However, DNNs trained by such dataset with co-occurrence bias may rely on wrong features while making decisions for classification. It will greatly affect the transferability of pre-trained DNNs. In this paper, we propose an interactive method to direct classifiers paying attentions to the regions that are manually specified by the users, in order to mitigate the influence of co-occurrence bias. We test on CelebA dataset, the pre-trained AlexNet is fine-tuned to focus on the specific facial attributes based on the …
While depth cameras and inertial sensors have been frequently leveraged for human action recognition, these sensing modalities are impractical in many scenarios where cost or environmental constraints prohibit their use. As such, there has been recent interest on human action recognition using low-cost, readily-available RGB cameras via deep convolutional neural networks. However, many of the deep convolutional neural networks proposed for action recognition thus far have relied heavily on learning global appearance cues directly from imaging data, resulting in highly complex network architect…
Xidian University（西安电子科技大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Saarland Informatics Campus（萨尔兰信息学园区）
Global average pooling (GAP) allows to localize discriminative information for recognition [40]. While GAP helps the convolution neural network to attend to the most discriminative features of an object, it may suffer if that information is missing e.g. due to camera viewpoint changes. To circumvent this issue, we argue that it is advantageous to attend to the global configuration of the object by modeling spatial relations among high-level features. We propose a novel architecture for Person Re-Identification, based on a novel parameter-free spatial attention layer introducing spatial relatio…
Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated data, where a high quality of annotation is often crucial. In this work, we propose to fine-tune CNNs for image retrieval on a large collection of unordered images in a fully automated manner. Reconstructed 3D models obtained by the state-of-the-art retrieval and structure-from-motion methods guide t…
When processing similar frames in succession, we can take advantage of the locality of the convolution operation to reevaluate only portions of the image that changed from the previous frame. By saving the output of a layer of convolutions and calculating the change from frame to frame, we can reuse previous activations and save computational resources that would otherwise be wasted recalculating convolutions whose outputs we have already observed. This technique can be applied to many domains, such as processing videos from stationary video cameras, studying the effects of occluding or distor…
Label distributions in camera-trap images are highly imbalanced and long-tailed, resulting in neural networks tending to be biased towards head-classes that appear frequently. Although long-tail learning has been extremely explored to address data imbalances, few studies have been conducted to consider camera-trap characteristics, such as multi-domain and multi-frame setup. Here, we propose a unified framework and introduce two datasets for long-tailed camera-trap recognition. We first design domain experts, where each expert learns to balance imperfect decision boundaries caused by data imbal…
What-and-Where to Match: Deep Spatially Multiplicative Integration Networks for Person Re-identification
The University of Queensland（昆士兰大学）；The University of New South Wales（新南威尔士大学）；The University of Sydney（悉尼大学）
在不相交的摄像机视图中匹配行人，称为人的重新识别（re-id），是一个具有挑战性的问题，对视觉识别和监控具有重要意义。大多数现有的方法利用空间操作中的局部区域来执行局部对应的匹配。然而，它们本质上从每个图像的预划分区域提取\emph{fixed}表示，并且随后基于所提取的表示执行匹配。对于这个管道中的模型，无法捕获对区分正对和负对至关重要的局部更精细的模式，从而使它们表现不佳。在本文中，我们提出了一种新的深度乘法积分门函数，它回答了有效人员re-id的\emph{what-and-where to match}问题。为了解决\emph{what}匹配问题，我们的深度网络通过以乘法方式学习联合表示来强调共同的局部模式。该网络包括两个卷积神经网络（CNN）来提取卷积激活，并生成用于行人匹配的相关描述符。因此，这导致成对图像的灵活表示。为了解决\emph{where}匹配问题，我们通过四方向递归神经网络执行空间递归池化来对抗空间未对准，以相对于整个图像在所有位置上施加空间依赖性。所提出的网络被设计为端到端可训练的，以空间对齐的方式表征局部成对特征交互。为了证明我们的方法的优越性，进行了广泛的实验在三个基准数据集：VIPeR，CUHK 03和Market-1501。
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）
Despite Neural Radiance Fields (NeRF) showing compelling results in photorealistic novel views synthesis of real-world scenes, most existing approaches require accurate prior camera poses. Although approaches for jointly recovering the radiance field and camera pose exist (BARF), they rely on a cumbersome coarse-to-fine auxiliary positional embedding to ensure good performance. We present Gaussian Activated neural Radiance Fields (GARF), a new positional embedding-free neural radiance field architecture - employing Gaussian activations - that outperforms the current state-of-the-art in terms o…
Absolute camera pose regressors estimate the position and orientation of a camera from the captured image alone. Typically, a convolutional backbone with a multi-layer perceptron head is trained with images and pose labels to embed a single reference scene at a time. Recently, this scheme was extended for learning multiple scenes by replacing the MLP head with a set of fully connected layers. In this work, we propose to learn multi-scene absolute camera pose regression with Transformers, where encoders are used to aggregate activation maps with self-attention and decoders transform latent feat…
MODAL: Multi-Modal Object Re-ID via Model-Driven Sparse Decoupling and Text-Image Differential Filtering
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）
