In this paper, we present a novel vision-based framework for tracking dynamic objects using guidance laws based on a rendezvous cone approach. These guidance laws enable an unmanned aircraft system equipped with a monocular camera to continuously follow a moving object within the sensor's field of view. We identify and classify feature point estimators for managing the occurrence of occlusions during the tracking process in an exclusive manner. Furthermore, we develop an open-source simulation environment and perform a series of simulations to show the efficacy of our methods.
In order to improve the precision of multi-robot SLAM multi-view target tracking process, a improved multi-robot SLAM multi-view target tracking algorithm based on panoramic vision in irregular environment was put forward, adding an correction factor to renew the existing Extended Kalman Filter (EKF) model, obtaining new coordinates X and Y after twice iterations. The paper has been accepted by Computing and Visualization in Science and this is a simplified version.
Tongji University（同济大学）；the University of Hong Kong（香港大学）
Vision-based object tracking has boosted extensive autonomous applications for unmanned aerial vehicles (UAVs). However, the dynamic changes in flight maneuver and viewpoint encountered in UAV tracking pose significant difficulties, e.g. , aspect ratio change, and scale variation. The conventional cross-correlation operation, while commonly used, has limitations in effectively capturing perceptual similarity and incorporates extraneous background information. To mitigate these limitations, this work presents a novel saliency-guided dynamic vision Transformer (SGDViT) for UAV tracking. The prop…
Tongji University（同济大学）；Tsinghua University（清华大学）；The University of Hong Kong（香港大学）
Although the manipulating of the unmanned aerial manipulator (UAM) has been widely studied, vision-based UAM approaching, which is crucial to the subsequent manipulating, generally lacks effective design. The key to the visual UAM approaching lies in object tracking, while current UAM tracking typically relies on costly model-based methods. Besides, UAM approaching often confronts more severe object scale variation issues, which makes it inappropriate to directly employ state-of-the-art model-free Siamese-based methods from the object tracking field. To address the above problems, this work pr…
The field of robotic vision is developing rapidly. Robots can react intelligently and provide assistance to user activities through sentient computing. Since industrial applications pose complex requirements that cannot be handled by humans, an efficient low cost and robust technique is required for the tracking of mobile industrial robots. The existing sensor based techniques for mobile robot tracking are expensive and complex to deploy, configure and maintain. Also some of them demand dedicated and often expensive hardware. This paper presents a low cost vision based technique called Hybrid …
Real-time tracking is an important problem in computer vision in which most methods are based on the conventional cameras. Neuromorphic vision is a concept defined by incorporating neuromorphic vision sensors such as silicon retinas in vision processing system. With the development of the silicon technology, asynchronous event-based silicon retinas that mimic neuro-biological architectures has been developed in recent years. In this work, we combine the vision tracking algorithm of computer vision with the information encoding mechanism of event-based sensors which is inspired from the neural …
Towards Real-World Aerial Vision Guidance with Categorical 6D Pose Tracker
Hunan University（湖南大学）；National Engineering Research Centre for Robot Visual Perception and Control（国家机器人视觉感知与控制技术工程研究中心）；Nanyang Technological University（南洋理工大学）
This paper proposes a concept of computer vision-based guidance assistance for agricultural vehicles to increase the accuracy in plowing and reduce driver's cognitive burden in long-lasting tillage operations. Plowing is a common agricultural practice to prepare the soil for planting in many countries and it can take place both in the spring and the fall. Since plowing operation requires high traction forces, it causes increased energy consumption. Moreover, longer operation time due to unnecessary maneuvers leads to higher fuel consumption. To provide necessary information for the driver and …
University of Technology Sydney（悉尼科技大学）；Imperial College London（帝国理工学院）
This work addresses the issue of motion compensation and pattern tracking in event camera data. An event camera generates asynchronous streams of events triggered independently by each of the pixels upon changes in the observed intensity. Providing great advantages in low-light and rapid-motion scenarios, such unconventional data present significant research challenges as traditional vision algorithms are not directly applicable to this sensing modality. The proposed method decomposes the tracking problem into a local SE(2) motion-compensation step followed by a homography registration of smal…
MARVL: Multi-Stage Guidance for Robotic Manipulation via Vision-Language Models
School of Intelligent Science and Technology, Nanjing University, China（南京大学智能科学与技术学院）；National Key Laboratory for Novel Software Technology, School of Artificial Intelligence, Nanjing University, China（南京大学新型软件技术国家实验室，人工智能学院）；School of Artificial Intelligence, Nanjing University, China（南京大学人工智能学院）；MACS Lab, University of Washington（华盛顿大学MACS实验室）
Tianjin University（天津大学）；University of South Carolina（南卡罗来纳大学）
Referring expression comprehension (REC) aims to localize a text-related region in a given image by a referring expression in natural language. Existing methods focus on how to build convincing visual and language representations independently, which may significantly isolate visual and language information. In this paper, we argue that for REC the referring expression and the target region are semantically correlated and subject, location and relationship consistency exist between vision and language.On top of this, we propose a novel approach called MutAtt to construct mutual guidance betwee…
Faculty of Electrical Engineering, Computer Science and Information Technology Osijek（奥西耶克电气工程、计算机科学与信息技术学院）
Visually impaired persons have significant problems in their everyday movement. Therefore, some of our previous work involves computer vision in developing assistance systems for guiding the visually impaired in critical situations. Some of those situations includes crosswalks on road crossings and stairs in indoor and outdoor environment. This paper presents an evaluation framework for computer vision-based guiding of the visually impaired persons in such critical situations. Presented framework includes the interface for labeling and storing referent human decisions for guiding directions an…
AREA3D: Active Reconstruction Agent with Unified Feed-Forward 3D Perception and Vision-Language Guidance
Southern University of Science and Technology（南方科技大学）；Harvard University（哈佛大学）；California Institute of Technology（加州理工学院）；Massachusetts Institute of Technology（麻省理工学院）
To address the SMC'17 data challenge -- "Data mining atomically resolved images for material properties", we first used the classic "blob detection" algorithms developed in computer vision to identify all atom centers in each STEM image frame. With the help of nearest neighbor analysis, we then found and labeled every atom center common to all the STEM frames and tracked their movements through the given time interval for both Molybdenum or Selenium atoms.
This paper describes the kinematics of the motion tracking of a rigid body using video recording. The novelty of the paper is on the adaptation of the methods and nomenclature used in Computer Vision to those used in Multibody System Dynamics. That way, the equations presented here can be used, for example, for inverse-dynamics multibody simulations driven by the motion tracking of selected bodies. This paper also adapts the well-known Zhang calibration method to the presented nomenclature.
A Survey of Fish Tracking Techniques Based on Computer Vision
College of Information and Electrical Engineering, China Agricultural University（中国农业大学信息与电气工程学院）；National Innovation Center for Digital Fishery, Ministry of Agriculture and Rural Affairs（农业农村部数字渔业创新中心）；College of Engineering, China Agricultural University（中国农业大学工学院）
The paper focuses on the problem of vision-based obstacle detection and tracking for unmanned aerial vehicle navigation. A real-time object localization and tracking strategy from monocular image sequences is developed by effectively integrating the object detection and tracking into a dynamic Kalman model. At the detection stage, the object of interest is automatically detected and localized from a saliency map computed via the image background connectivity cue at each frame; at the tracking stage, a Kalman filter is employed to provide a coarse prediction of the object state, which is furthe…
While the Vision Transformer (VT) architecture is becoming trendy in computer vision, pure VT models perform poorly on tiny datasets. To address this issue, this paper proposes the locality guidance for improving the performance of VTs on tiny datasets. We first analyze that the local information, which is of great importance for understanding images, is hard to be learned with limited data due to the high flexibility and intrinsic globality of the self-attention mechanism in VTs. To facilitate local information, we realize the locality guidance for VTs by imitating the features of an already …
Instituto Nacional de Astrofísica, Óptica y Electrónica（国家天体物理、光学与电子学研究所）；Escuela Politécnica Nacional（国立理工学院）
This paper describes the development of an embedded vision system for detection, location, and tracking of a color object; it makes use of a single 32-bit microprocessor to acquire image data, process, and perform actions according to the interpreted data. The system is intended for applications that need to make use of artificial vision for detection, location and tracking of a color object and its objective is to have achieve at reduced terms of size, power consumption, and cost.
University of Udine（乌迪内大学）；University of Catania（卡塔尼亚大学）
Understanding human-object interactions is fundamental in First Person Vision (FPV). Tracking algorithms which follow the objects manipulated by the camera wearer can provide useful cues to effectively model such interactions. Visual tracking solutions available in the computer vision literature have significantly improved their performance in the last years for a large variety of target objects and tracking scenarios. However, despite a few previous attempts to exploit trackers in FPV applications, a methodical analysis of the performance of state-of-the-art trackers in this domain is still m…
Istituto Italiano di Tecnologia（意大利技术研究院）；Politecnico di Milano（米兰理工大学）
Simulation tools are essential for robotics research, especially for those domains in which safety is crucial, such as Human-Robot Collaboration (HRC). However, it is challenging to simulate human behaviors, and existing robotics simulators do not integrate functional human models. This work presents Open-VICO, an open-source toolkit to integrate virtual human models in Gazebo focusing on vision-based human tracking. In particular, Open-VICO allows to combine in the same simulation environment realistic human kinematic models, multi-camera vision setups, and human-tracking techniques along wit…
Freie Universität Berlin（柏林自由大学）；University of Applied Science Kempten（肯普滕应用科学大学）；Humboldt-Universität zu Berlin（柏林洪堡大学）；Leibniz-Institute of Freshwater Ecology and Inland Fisheries（莱布尼茨淡水生态与内陆渔业研究所）
The study of animal behavior increasingly relies on (semi-) automatic methods for the extraction of relevant behavioral features from video or picture data. To date, several specialized software products exist to detect and track animals' positions in simple (laboratory) environments. Tracking animals in their natural environments, however, often requires substantial customization of the image processing algorithms to the problem-specific image characteristics. Here we introduce BioTracker, an open-source computer vision framework, that provides programmers with core functionalities that are e…
Active Spatial Guidance: Eliminating Injected Positional Mechanisms in Vision Transformers
School of Engineering, Yancheng Institute of Technology（盐城工学院工程学院）；School of Computer Information Engineering, Changzhou Institute of Technology（常州工学院计算机信息工程学院）；Advanced Robotics and Intelligent Systems Laboratory, School of Engineering, University of Guelph（圭尔夫大学工程学院先进机器人与智能系统实验室）
University of Tübingen（图宾根大学）；DeepMind（深度思维）；MPI for Intelligent Systems（马克斯·普朗克智能系统研究所）
Deep Metric Learning (DML) proposes to learn metric spaces which encode semantic similarities as embedding space distances. These spaces should be transferable to classes beyond those seen during training. Commonly, DML methods task networks to solve contrastive ranking tasks defined over binary class assignments. However, such approaches ignore higher-level semantic relations between the actual classes. This causes learned embedding spaces to encode incomplete semantic context and misrepresent the semantic relation between classes, impacting the generalizability of the learned metric space. T…
Unmanned aerial vehicles (UAV) have been widely used in various fields, and their invasion of security and privacy has aroused social concern. Several detection and tracking systems for UAVs have been introduced in recent years, but most of them are based on radio frequency, radar, and other media. We assume that the field of computer vision is mature enough to detect and track invading UAVs. Thus we propose a visible light mode dataset called Dalian University of Technology Anti-UAV dataset, DUT Anti-UAV for short. It contains a detection dataset with a total of 10,000 images and a tracking d…
FalconTrack: Photorealistic Auto-Labeled Perception and Physics-Aware Vision-Based Aerial Tracking
Department of Electrical and Computer Engineering, University of Illinois at Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校电子与计算机工程系）；Toyota Motor North America R&D（丰田北美研发）
General Vision Encoder Features as Guidance in Medical Image Registration
Klinikum Rechts der Isar, Technical University of Munich（慕尼黑工业大学右岸医院）；School of Computation, Information & Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（亥姆霍兹慕尼黑中心生物医学影像机器学习研究所）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Cancer Research UK Cambridge Institute, University of Cambridge（剑桥大学英国癌症研究剑桥研究所）；Department of Oncology, University of Cambridge（剑桥大学肿瘤学系）；German Cancer Consortium (DKTK), Partner Site Munich（德国癌症联盟慕尼黑合作站点）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）
3D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidance
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；Meituan Inc.（美团公司）；University of Trento（特伦托大学）；Peking University（北京大学）；Carnegie Mellon University（卡内基梅隆大学）
Visual Computing Institute, RWTH Aachen University（亚琛工业大学视觉计算研究所）
The most common paradigm for vision-based multi-object tracking is tracking-by-detection, due to the availability of reliable detectors for several important object categories such as cars and pedestrians. However, future mobile systems will need a capability to cope with rich human-made environments, in which obtaining detectors for every possible object category would be infeasible. In this paper, we propose a model-free multi-object tracking approach that uses a category-agnostic image segmentation method to track objects. We present an efficient segmentation mask-based tracker which associ…
Efficient Language-to-Vision Feature Injection for Referring Single-Object Tracking
Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间应用工程与技术中心）；University of Chinese Academy of Sciences（中国科学院大学）
University of Udine（乌迪内大学）；University of Catania（卡塔尼亚大学）
Understanding human-object interactions is fundamental in First Person Vision (FPV). Tracking algorithms which follow the objects manipulated by the camera wearer can provide useful cues to effectively model such interactions. Despite a few previous attempts to exploit trackers in FPV applications, a methodical analysis of the performance of state-of-the-art visual trackers in this domain is still missing. In this short paper, we provide a recap of the first systematic study of object tracking in FPV. Our work extensively analyses the performance of recent and baseline FPV trackers with respec…
Mobile Augmented Reality (MAR) is becoming an important cyber-physical system application given the ubiquitous availability of mobile phones. With the need to operate in unprepared environments, accurate and robust registration and tracking has become an important research problem to solve. In fact, when MAR is used for tele-interactive applications involving large distances, say from an accident site to insurance office, tracking at both the ends is desirable and further it is essential to appropriately fuse inertial and vision sensors data. In this paper, we present results and discuss some …
Beijing Jiaotong University（北京交通大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of North Texas（北得克萨斯大学）
Relying on Transformer for complex visual feature learning, object tracking has witnessed the new standard for state-of-the-arts (SOTAs). However, this advancement accompanies by larger training data and longer training period, making tracking increasingly expensive. In this paper, we demonstrate that the Transformer-reliance is not necessary and the pure ConvNets are still competitive and even better yet more economical and friendly in achieving SOTA tracking. Our solution is to unleash the power of multimodal vision-language (VL) tracking, simply using ConvNets. The essence lies in learning …
Autonomous ground vehicles have been designed for the purpose of that relies on ranging and bearing information received from forward looking camera on the Formation control . A visual guidance control algorithm is designed where real time image processing is used to provide feedback signals. The vision subsystem and control subsystem work in parallel to accomplish formation control. A proportional navigation and line of sight guidance laws are used to estimate the range and bearing information from the leader vehicle using the vision subsystem. The algorithms for vision detection and localiza…
Hong Kong University of Science and Technology（香港科技大学）
We propose a stereo vision-based approach for tracking the camera ego-motion and 3D semantic objects in dynamic autonomous driving scenarios. Instead of directly regressing the 3D bounding box using end-to-end approaches, we propose to use the easy-to-labeled 2D detection and discrete viewpoint classification together with a light-weight semantic inference method to obtain rough 3D object measurements. Based on the object-aware-aided camera pose tracking which is robust in dynamic environments, in combination with our novel dynamic object bundle adjustment (BA) approach to fuse temporal sparse…
