Surface reconstruction from multiple, calibrated images is a challenging task - often requiring a large number of collected images with significant overlap. We look at the specific case of human foot reconstruction. As with previous successful foot reconstruction work, we seek to extract rich per-pixel geometry cues from multi-view RGB images, and fuse these into a final 3D object. Our method, FOCUS, tackles this problem with 3 main contributions: (i) SynFoot2, an extension of an existing synthetic foot dataset to include a new data type: dense correspondence with the parameterized foot model FIND; (ii) an uncertainty-aware dense correspondence predictor trained on our synthetic dataset; (iii) two methods for reconstructing a 3D surface from dense correspondence predictions: one inspired by Structure-from-Motion, and one optimization-based using the FIND model. We show that our reconstruction achieves state-of-the-art reconstruction quality in a few-view setting, performing comparably to state-of-the-art when many views are available, and runs substantially faster. We release our synthetic dataset to the research community. Code is available at: https://github.com/OllieBoyne/FOCUS
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）
In this paper, we propose a novel method to register football broadcast video frames on the static top view model of the playing surface. The proposed method is fully automatic in contrast to the current state of the art which requires manual initialization of point correspondences between the image and the static model. Automatic registration using existing approaches has been difficult due to the lack of sufficient point correspondences. We investigate an alternate approach exploiting the edge information from the line markings on the field. We formulate the registration problem as a nearest…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
Model compression and knowledge distillation have been successfully applied for cross-architecture and cross-domain transfer learning. However, a key requirement is that training examples are in correspondence across the domains. We show that in many scenarios of practical importance such aligned data can be synthetically generated using computer graphics pipelines allowing domain adaptation through distillation. We apply this technique to learn models for recognizing low-resolution images using labeled high-resolution images, non-localized objects using labeled localized objects, line-drawing…
Self-supervised audio-visual learning aims to capture useful representations of video by leveraging correspondences between visual and audio inputs. Existing approaches have focused primarily on matching semantic information between the sensory streams. We propose a novel self-supervised task to leverage an orthogonal principle: matching spatial information in the audio stream to the positions of sound sources in the visual stream. Our approach is simple yet effective. We train a model to determine whether the left and right audio channels have been flipped, forcing it to reason about spatial …
Estimating the head pose of a person is a crucial problem that has a large amount of applications such as aiding in gaze estimation, modeling attention, fitting 3D models to video and performing face alignment. Traditionally head pose is computed by estimating some keypoints from the target face and solving the 2D to 3D correspondence problem with a mean human head model. We argue that this is a fragile method because it relies entirely on landmark detection performance, the extraneous head model and an ad-hoc fitting step. We present an elegant and robust way to determine pose by training a m…
École Normale Supérieure（巴黎高等师范学院）；CNRS（法国国家科学研究中心）；PSL Research University（巴黎文理研究大学）；Czech Technical University in Prague（捷克布拉格技术大学）；Inria（法国国家信息与自动化研究所）；DeepMind
In this work we target the problem of estimating accurately localised correspondences between a pair of images. We adopt the recent Neighbourhood Consensus Networks that have demonstrated promising performance for difficult correspondence problems and propose modifications to overcome their main limitations: large memory consumption, large inference time and poorly localised correspondences. Our proposed modifications can reduce the memory footprint and execution time more than $10\times$, with equivalent results. This is achieved by sparsifying the correlation tensor containing tentative matc…
George Mason University（乔治梅森大学）；Siemens Corporate Technology（西门子企业技术研究院）
We consider the problem of 3D object pose estimation. While much recent work has focused on the RGB domain, the reliance on accurately annotated images limits their generalizability and scalability. On the other hand, the easily available CAD models of objects are rich sources of data, providing a large number of synthetically rendered images. In this paper, we solve this key problem of existing methods requiring expensive 3D pose annotations by proposing a new method that matches RGB images to CAD models for object pose estimation. Our key innovations compared to existing work include removin…
DenVisCoM: Dense Vision Correspondence Mamba for Efficient and Real-time Optical Flow and Stereo Estimation
Machine Intelligence Group, Birla Institute of Technology and Science, Pilani, Hyderabad Campus, India（机智人工智能组，比拉理工学院，比拉理工学院，海得拉巴校区，印度）；Université Côte d’Azur, Inria, France（蔚蓝大学，法国国家信息与自动化研究所）
Spatio-Temporal Garment Reconstruction Using Diffusion Mapping via Pattern Coordinates
CVLab, École Polytechnique Fédérale de Lausanne (EPFL)（瑞士联邦理工学院（EPFL）计算机视觉实验室）；Mohamed bin Zayed University of Artificial Intelligence（马尔代夫人工智能大学）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）；Pinscreen and Mohamed bin Zayed University of Artificial Intelligence（Pinscreen和马尔代夫人工智能大学）
TrackFish3D: Self-Supervised 3D Tracking of Schooling Fish from Multi-view Videos
The University of Hong Kong（香港大学）；Centre for Transformative Garment Production（转型服装生产中心）；Hong Kong University of Science and Technology（香港科技大学）；Texas A&M University（德克萨斯A&M大学）
Perturbation-Regularized Open-Vocabulary Remote Sensing Segmentation with Unified Multi-Domain Evaluation
Department of Electronic Engineering and Information Science, University of Science and Technology of China（电子工程与信息科学系，中国科学技术大学）；School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University（人工智能学院，光学与电子学系（iOPEN），西北工业大学）；Institute of Artificial Intelligence (TeleAI), China Telecom, P. R. China（人工智能研究院（TeleAI），中国电信，中华人民共和国）
OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shanghai AI Lab（上海人工智能实验室）；Beihang University（北京航空航天大学）；Nanyang Technological University（南洋理工大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong（香港中文大学）
