A panoramic image mosaic is an attractive visualization for viewing many overlapping photos, but its images must be both captured and processed correctly to produce an acceptable composite. We propose Swipe Mosaics, an interactive visualization that places the individual video frames on a 2D planar map that represents the layout of the physical scene. Compared to traditional panoramic mosaics, our capture is easier because the user can both translate the camera center and film moving subjects. Processing and display degrade gracefully if the footage lacks distinct, overlapping, non-repeating texture. Our proposed visual odometry algorithm produces a distribution over (x,y) translations for image pairs. Inferring a distribution of possible camera motions allows us to better cope with parallax, lack of texture, dynamic scenes, and other phenomena that hurt deterministic reconstruction techniques. Robustness is obtained by training on synthetic scenes with known camera motions. We show that Swipe Mosaics are easy to generate, support a wide range of difficult scenes, and are useful for documenting a scene for closer inspection.
In this paper, we present feature-based technique for construction of mosaic image from underwater video sequence, which suffers from parallax distortion due to propagation properties of light in the underwater environment. The most of the available mosaic tools and underwater image mosaicing techniques yields final result with some artifacts such as blurring, ghosting and seam due to presence of parallax in the input images. The removal of parallax from input images may not reduce its effects instead it must be corrected in successive steps of mosaicing. Thus, our approach minimizes the paral…
MosaicMem: Hybrid Spatial Memory for Controllable Video World Models
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；The University of Osaka（大阪大学）；Georgia Institute of Technology（佐治亚理工学院）；Mujin Inc.（Mujin公司）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Taiyuan University of Technology（太原本科技大学）
Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School（马蒂诺斯生物医学成像中心、麻省总医院、哈佛医学院）；Computer Science and Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室（CSAIL））；Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London（伦敦大学学院惠康/EPSRC介入与外科科学中心）；Centre for Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；KU Leuven（鲁汶大学）；King’s College London（伦敦国王学院）
Video mosaicking requires the registration of overlapping frames located at distant timepoints in the sequence to ensure global consistency of the reconstructed scene. However, fully automated registration of such long-range pairs is (i) challenging when the registration of images itself is difficult; and (ii) computationally expensive for long sequences due to the large number of candidate pairs for registration. In this paper, we introduce an efficient framework for the active annotation of long-range pairwise correspondences in a sequence. Our framework suggests pairs of images that are sou…
Graduate School in Advanced Optical Technologies (SAOT)（高级光学技术研究生院（SAOT））
The acquisition of high-resolution retinal fundus images with a large field of view (FOV) is challenging due to technological, physiological and economic reasons. This paper proposes a fully automatic framework to reconstruct retinal images of high spatial resolution and increased FOV from multiple low-resolution images captured with non-mydriatic, mobile and video-capable but low-cost cameras. Within the scope of one examination, we scan different regions on the retina by exploiting eye motion conducted by a patient guidance. Appropriate views for our mosaicing method are selected based on op…
Institute for Computer Science（计算机科学研究所）；Cologne University of Applied Sciences（科隆应用技术大学）
This technical report describes an improved image mosaicking algorithm. It is based on Jain's logarithmic search algorithm [Jain 1981] which is coupled to the method of Kourogi (1999} for matching images in a video sequence. Logarithmic search has a better invariance against illumination changes than the original optical-flow-based method of Kourogi.
Motion-Saliency Complementary Masked Modeling for Point Cloud Video Understanding
School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；State Key Laboratory of Intelligent Geotechnics and Tunnelling (FSDI)（智能岩土与隧道工程国家重点实验室）；School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；School of Information and Communication Engineering, Xi’an Jiaotong University（西安交通大学信息与通信工程学院）
University College London（伦敦大学学院）；KU Leuven University（鲁汶大学）；University Hospital Leuven（鲁汶大学医院）
Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time. Deep learning techniques can facilitate in ov…
Institute for Systems and Robotics（系统与机器人研究所）；Instituto Superior Técnico（高等技术学院）；The Boston Consulting Group（波士顿咨询集团）
The majority of the approaches to the automatic recovery of a panoramic image from a set of partial views are suboptimal in the sense that the input images are aligned, or registered, pair by pair, e.g., consecutive frames of a video clip. These approaches lead to propagation errors that may be very severe, particularly when dealing with videos that show the same region at disjoint time intervals. Although some authors have proposed a post-processing step to reduce the registration errors in these situations, there have not been attempts to compute the optimal solution, i.e., the registrations…
The problem of overlapping occlusion in target recognition has been a difficult research problem, and the situation of mutual occlusion of ship targets in narrow waters still exists. In this paper, an improved mosaic data enhancement method is proposed, which optimizes the reading method of the data set, strengthens the learning ability of the detection algorithm for local features, improves the recognition accuracy of overlapping targets while keeping the test speed unchanged, reduces the decay rate of recognition ability under different resolutions, and strengthens the robustness of the algo…
We present FacialFilmroll, a solution for spatially and temporally consistent editing of faces in one or multiple shots. We build upon unwrap mosaic [Rav-Acha et al. 2008] by specializing it to faces. We leverage recent techniques to fit a 3D face model on monocular videos to (i) improve the quality of the mosaic for edition and (ii) permit the automatic transfer of edits from one shot to other shots of the same actor. We explain how FacialFilmroll is integrated in post-production facility. Finally, we present video editing results using FacialFilmroll on high resolution videos.
Centre de Recherche en Automatique de Nancy（南锡自动化研究中心）；Nancy-University（南锡大学）；CNRS（法国国家科学研究中心）
Bladder cancer is widely spread in the world. Many adequate diagnosis techniques exist. Video-endoscopy remains the standard clinical procedure for visual exploration of the bladder internal surface. However, video-endoscopy presents the limit that the imaged area for each image is about nearly 1cm2. And, lesions are, typically, spread over several images. The aim of this contribution is to assess the performance of two mosaicing algorithms leading to the construction of panoramic maps (one unique image) of bladder walls. The quantitative comparison study is performed on a set of real endoscop…
Method for a mosaic image representation (MIR) is proposed for a selective treatment of image fragments of different transition frequency. MIR method is based on piecewise-constant image approximation on a non-uniform orthogonal grid constructed by the following recurrent multigrid algorithm. A sequence of nested uniform grids is built, such that each cell of a current grid is subdivided into four smaller cells for the next grid designing. In each grid the cells are selected, where the color intensity function can be approximated by its average value with a given precision (thereafter 'good' c…
This paper introduces a novel method to simultaneously super-resolve and colour-predict images acquired by snapshot mosaic sensors. These sensors allow for spectral images to be acquired using low-power, small form factor, solid-state CMOS sensors that can operate at video frame rates without the need for complex optical setups. Despite their desirable traits, their main drawback stems from the fact that the spatial resolution of the imagery acquired by these sensors is low. Moreover, chromatic mapping in snapshot mosaic sensors is not straightforward since the bands delivered by the sensor te…
The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；IIIT Delhi（德里印度理工学院）
The possibility of sharing one's point of view makes use of wearable cameras compelling. These videos are often long, boring and coupled with extreme shake, as the camera is worn on a moving person. Fast forwarding (i.e. frame sampling) is a natural choice for quick video browsing. However, this accentuates the shake caused by natural head motion in an egocentric video, making the fast forwarded video useless. We propose EgoSampling, an adaptive frame sampling that gives stable, fast forwarded, hyperlapse videos. Adaptive frame sampling is formulated as an energy minimization problem, whose op…
