Learning to See by Moving

Improved Handling of Motion Blur in Online Object Detection

We wish to detect specific categories of objects, for online vision systems that will run in the real world. Object detection is already very challenging. It is even harder when the images are blurred, from the camera being in a car or a hand-held phone. Most existing efforts either focused on sharp images, with easy to label ground truth, or they have treated motion blur as one of many generic corruptions. Instead, we focus especially on the details of egomotion induced blur. We explore five classes of remedies, where each targets different potential causes for the performance gap between sha…

用于大气湍流中目标分割的无监督区域生长网络

Unsupervised Region-Growing Network for Object Segmentation in Atmospheric Turbulence

提出无监督“先检测后生长”区域生长网络，利用刚性几何一致性、Sampson距离及时空损失，在大气湍流视频中鲁棒分割运动目标，并发布首个真实远距离湍流真值数据集。

Moving Object Detection in Video Using Saliency Map and Subspace Learning

Moving object detection is a key to intelligent video analysis. On the one hand, what moves is not only interesting objects but also noise and cluttered background. On the other hand, moving objects without rich texture are prone not to be detected. So there are undesirable false alarms and missed alarms in many algorithms of moving object detection. To reduce the false alarms and missed alarms, in this paper, we propose to incorporate a saliency map into an incremental subspace analysis framework where the saliency map makes estimated background has less chance than foreground (i.e., moving o…

Towards 3D Visualization of Video from Frames

We explain theoretically how to reconstruct the 3D scene from successive frames in order to see the video in 3D. To do this, features, associated to moving rigid objects in 3D, are extracted in frames and matched. The vanishing point computed in frame corresponding to the direction of moving object is used for 3D positioning of the 3D structure of the moving object. First experiments are conducted and the obtained results are shown and publicly available. They demonstrate the feasibility of our method. We conclude this paper by future works in order to improve this method tacking into account …

Learn to See Faster: Pushing the Limits of High-Speed Camera with Deep Underexposed Image Denoising

The ability to record high-fidelity videos at high acquisition rates is central to the study of fast moving phenomena. The difficulty of imaging fast moving scenes lies in a trade-off between motion blur and underexposure noise: On the one hand, recordings with long exposure times suffer from motion blur effects caused by movements in the recorded scene. On the other hand, the amount of light reaching camera photosensors decreases with exposure times so that short-exposure recordings suffer from underexposure noise. In this paper, we propose to address this trade-off by treating the problem of…

Consistent Depth of Moving Objects in Video

We present a method to estimate depth of a dynamic scene, containing arbitrary moving objects, from an ordinary video captured with a moving camera. We seek a geometrically and temporally consistent solution to this underconstrained problem: the depth predictions of corresponding points across frames should induce plausible, smooth motion in 3D. We formulate this objective in a new test-time training framework where a depth-prediction CNN is trained in tandem with an auxiliary scene-flow prediction MLP over the entire input video. By recursively unrolling the scene-flow prediction MLP over var…

Memristive Threshold Logic Circuit Design of Fast Moving Object Detection

Real-time detection of moving objects involves memorisation of features in the template image and their comparison with those in the test image. At high sampling rates, such techniques face the problems of high algorithmic complexity and component delays. We present a new resistive switching based threshold logic cell which encodes the pixels of a template image. The cell comprises a voltage divider circuit that programs the resistances of the memristors arranged in a single node threshold logic network and the output is encoded as a binary value using a CMOS inverter gate. When a test image i…

基于在线结构化稀疏的卫星视频运动目标检测

Online Structured Sparsity-based Moving Object Detection from Satellite Videos

针对批量式低秩结构化稀疏分解检测卫星视频运动目标存在延迟的问题，提出逐帧交替更新的在线O-LSD方法，理论保证收敛，实验显示其性能与批量法相当且延迟大幅降低。

DyStaB: Unsupervised Object Segmentation via Dynamic-Static Bootstrapping

We describe an unsupervised method to detect and segment portions of images of live scenes that, at some point in time, are seen moving as a coherent whole, which we refer to as objects. Our method first partitions the motion field by minimizing the mutual information between segments. Then, it uses the segments to learn object models that can be used for detection in a static image. Static and dynamic models are represented by deep neural networks trained jointly in a bootstrapping strategy, which enables extrapolation to previously unseen objects. While the training process requires motion, …

Unsupervised object-centric video generation and decomposition in 3D

A natural approach to generative modeling of videos is to represent them as a composition of moving objects. Recent works model a set of 2D sprites over a slowly-varying background, but without considering the underlying 3D scene that gives rise to them. We instead propose to model a video as the view seen while moving through a scene with multiple 3D objects and a 3D background. Our model is trained from monocular videos without any supervision, yet learns to generate coherent 3D scenes containing several moving objects. We conduct detailed experiments on two datasets, going beyond the visual…