Animal pose estimation has recently come into the limelight due to its application in biology, zoology, and aquaculture. Deep learning methods have effectively been applied to human pose estimation. However, the major bottleneck to the application of these methods to animal pose estimation is the unavailability of sufficient quantities of labeled data. Though there are ample quantities of unlabelled data publicly available, it is economically impractical to label large quantities of data for each animal. In addition, due to the wide variety of body shapes in the animal kingdom, the transfer of knowledge across domains is ineffective. Given the fact that the human brain is able to recognize animal pose without requiring large amounts of labeled data, it is only reasonable that we exploit unsupervised learning to tackle the problem of animal pose recognition from the available, unlabelled data. In this paper, we introduce a novel architecture that is able to recognize the pose of multiple animals fromunlabelled data. We do this by (1) removing background information from each image and employing an edge detection algorithm on the body of the animal, (2) Tracking motion of the edge pixels and performing agglomerative clustering to segment body parts, (3) employing contrastive learning to discourage grouping of distant body parts together. Hence we are able to distinguish between body parts of the animal, based on their visual behavior, instead of the underlying anatomy. Thus, we are able to achieve a more effective classification of the data than their human-labeled counterparts. We test our model on the TigDog and WLD (WildLife Documentary) datasets, where we outperform state-of-the-art approaches by a significant margin. We also study the performance of our model on other public data to demonstrate the generalization ability of our model.
Classification and identification of wild animals for tracking and protection purposes has become increasingly important with the deterioration of the environment, and technology is the agent of change which augments this process with novel solutions. Computer vision is one such technology which uses the abilities of artificial intelligence and machine learning models on visual inputs. Convolution neural networks (CNNs) have multiple layers which have different weights for the purpose of prediction of a particular input. The precedent for classification, however, is set by the image processing…
The University of Sydney（悉尼大学）；Xidian University（西安电子科技大学）；JD Explore Academy（京东探索研究院）
Accurate animal pose estimation is an essential step towards understanding animal behavior, and can potentially benefit many downstream applications, such as wildlife conservation. Previous works only focus on specific animals while ignoring the diversity of animal species, limiting the generalization ability. In this paper, we propose AP-10K, the first large-scale benchmark for mammal animal pose estimation, to facilitate the research in animal pose estimation. AP-10K consists of 10,015 images collected and filtered from 23 animal families and 54 species following the taxonomic rank and high-…
The University of Osaka（大阪大学）；Kyoto University（京都大学）；Tokyo Institute of Technology（东京技术大学）；National Institute of Advanced Industrial Science and Technology (AIST)（国家先进工业科学与技术研究院）；Visual Geometry Group, University of Oxford（牛津大学视觉几何组）
Empowering Wildlife Guardians: An Equitable Digital Stewardship and Reward System for Biodiversity Conservation using Deep Learning and 3/4G Camera Traps
Liverpool John Moores University（利物浦约翰摩尔斯大学）；Gap Africa Projects（Gap非洲项目）；Welgevonden Game Reserve（韦尔格翁登禁猎区）；Borneo Futures（婆罗洲未来组织）；Czech Technical University（捷克技术大学）
Camera traps are a proven tool in biology and specifically biodiversity research. However, camera traps including depth estimation are not widely deployed, despite providing valuable context about the scene and facilitating the automation of previously laborious manual ecological methods. In this study, we propose an automated camera trap-based approach to detect and identify animals using depth estimation. To detect and identify individual animals, we propose a novel method D-Mask R-CNN for the so-called instance segmentation which is a deep learning-based technique to detect and delineate ea…
Ove Arup and Partners（奥雅纳工程顾问公司）；Newcastle University（纽卡斯尔大学）
Understanding the abundance of a species is the first step towards understanding both its long-term sustainability and the impact that we may be having upon it. Ecologists use camera traps to remotely survey for the presence of specific animal species. Previous studies have shown that deep learning models can be trained to automatically detect and classify animals within camera trap imagery with high levels of confidence. However, the ability to train these models is reliant upon having enough high-quality training data. What happens when the animal is rare or the data sets are non-existent? T…
Improving Wildlife Out-of-Distribution Detection: Africas Big Five
Institute for Artificial Intelligent Systems（人工智能系统研究所）；University of Johannesburg（约翰内斯堡大学）；Department of Electrical Engineering（电气工程系）；Linköping University（利马大学）
Computer Vision for Primate Behavior Analysis in the Wild
German Primate Center, Leibniz Institute for Primate Research（德国灵长类动物中心，莱布尼茨灵长类动物研究所）；Max Planck Institute for Dynamics and Self-Organization（马克斯·普朗克动力学与自组织研究所）
ImageNet-1k is a dataset often used for benchmarking machine learning (ML) models and evaluating tasks such as image recognition and object detection. Wild animals make up 27% of ImageNet-1k but, unlike classes representing people and objects, these data have not been closely scrutinized. In the current paper, we analyze the 13,450 images from 269 classes that represent wild animals in the ImageNet-1k validation set, with the participation of expert ecologists. We find that many of the classes are ill-defined or overlapping, and that 12% of the images are incorrectly labeled, with some classes…
Knowledge over the number of animals in large wildlife reserves is a vital necessity for park rangers in their efforts to protect endangered species. Manual animal censuses are dangerous and expensive, hence Unmanned Aerial Vehicles (UAVs) with consumer level digital cameras are becoming a popular alternative tool to estimate livestock. Several works have been proposed that semi-automatically process UAV images to detect animals, of which some employ Convolutional Neural Networks (CNNs), a recent family of deep learning algorithms that proved very effective in object detection in large dataset…
SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild
University of Cambridge（剑桥大学）；Southern University of Science and Technology（南方科技大学）；Tsinghua University（清华大学）；IMATI-CNR, Milan, Italy（意大利米兰IMATI-CNR研究所）
本文提出SAM 3D Animal，首个可提示的多动物3D重建框架，基于SMAL+参数化动物模型，通过关键点和掩码提示实现拥挤和遮挡场景的可靠重建，利用Herd3D数据集在多个数据集上取得最优结果。
CHIRP dataset: towards long-term, individual-level, behavioral monitoring of bird populations in the wild
Centre for the Advanced Study of Collective Behaviour, University of Konstanz（康斯坦茨大学集体行为高级研究中心）；Dept. of Collective Behavior, Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所集体行为系）；Dept. of Biology, University of Konstanz（康斯坦茨大学生物学系）；Dept. of Computer and Information Science, University of Konstanz（康斯坦茨大学计算机与信息科学系）；Luondua Boreal Field Station（卢昂杜阿北方野外站）；School of Biological and Medical Sciences, Oxford Brookes University（牛津布鲁克斯大学生物与医学科学学院）；Dept. of Zoology, Stockholm University（斯德哥尔摩大学动物学系）；Dept. of Ecology of Animal Societies, Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所动物社会生态学系）
We propose a novel end-to-end curriculum learning approach for sparsely labelled animal datasets leveraging large volumes of unlabelled data to improve supervised species detectors. We exemplify the method in detail on the task of finding great apes in camera trap footage taken in challenging real-world jungle environments. In contrast to previous semi-supervised methods, our approach adjusts learning parameters dynamically over time and gradually improves detection quality by steering training towards virtuous self-reinforcement. To achieve this, we propose integrating pseudo-labelling with c…
Universitat de Barcelona（巴塞罗那大学）；Universidade Federal de Santa Catarina（圣卡塔琳娜联邦大学）；Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；Universidade Federal do Paraná（巴拉那联邦大学）；Witwatersrand University（威特沃特斯兰德大学）；Oregon State University（俄勒冈州立大学）；University of Zurich（苏黎世大学）；Universitat Autonoma de Barcelona（巴塞罗那自治大学）
Effective conservation actions require effective population monitoring. However, accurately counting animals in the wild to inform conservation decision-making is difficult. Monitoring populations through image sampling has made data collection cheaper, wide-reaching and less intrusive but created a need to process and analyse this data efficiently. Counting animals from such data is challenging, particularly when densely packed in noisy images. Attempting this manually is slow and expensive, while traditional computer vision methods are limited in their generalisability. Deep learning is the …
Universidad de Antioquia UdeA（安蒂奥基亚大学（UdeA））；Instituto Tecnológico Metropolitano ITM（大都会理工学院（ITM））
Non intrusive monitoring of animals in the wild is possible using camera trapping framework, which uses cameras triggered by sensors to take a burst of images of animals in their habitat. However camera trapping framework produces a high volume of data (in the order on thousands or millions of images), which must be analyzed by a human expert. In this work, a method for animal species identification in the wild using very deep convolutional neural networks is presented. Multiple versions of the Snapshot Serengeti dataset were used in order to probe the ability of the method to cope with differ…
We present an Active Learning (AL) strategy for re-using a deep Convolutional Neural Network (CNN)-based object detector on a new dataset. This is of particular interest for wildlife conservation: given a set of images acquired with an Unmanned Aerial Vehicle (UAV) and manually labeled gound truth, our goal is to train an animal detector that can be re-used for repeated acquisitions, e.g. in follow-up years. Domain shifts between datasets typically prevent such a direct model application. We thus propose to bridge this gap using AL and introduce a new criterion called Transfer Sampling (TS). T…
Visual animal biometrics is rapidly gaining popularity as it enables a non-invasive and cost-effective approach for wildlife monitoring applications. Widespread usage of camera traps has led to large volumes of collected images, making manual processing of visual content hard to manage. In this work, we develop a framework for automatic detection and recognition of individuals in different patterned species like tigers, zebras and jaguars. Most existing systems primarily rely on manual input for localizing the animal, which does not scale well to large datasets. In order to automate the detect…
Camera traps, unmanned observation devices, and deep learning-based image recognition systems have greatly reduced human effort in collecting and analyzing wildlife images. However, data collected via above apparatus exhibits 1) long-tailed and 2) open-ended distribution problems. To tackle the open-set long-tailed recognition problem, we propose the Temporal Flow Mask Attention Network that comprises three key building blocks: 1) an optical flow module, 2) an attention residual module, and 3) a meta-embedding classifier. We extract temporal features of sequential frames using the optical flow…
Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing Institute of Software Technology（南京软件技术研究院）；University of North Texas（北德克萨斯大学）
Multi-animal tracking (MAT), a multi-object tracking (MOT) problem, is crucial for animal motion and behavior analysis and has many crucial applications such as biology, ecology and animal conservation. Despite its importance, MAT is largely under-explored compared to other MOT problems such as multi-human tracking due to the scarcity of dedicated benchmarks. To address this problem, we introduce AnimalTrack, a dedicated benchmark for multi-animal tracking in the wild. Specifically, AnimalTrack consists of 58 sequences from a diverse selection of 10 common animal categories. On average, each s…
Politecnico di Milano（米兰理工大学）；University of California, Davis（加州大学戴维斯分校）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Activity recognition and, more generally, behavior inference tasks are gaining a lot of interest. Much of it is work in the context of human behavior. New available tracking technologies for wild animals are generating datasets that indirectly may provide information about animal behavior. In this work, we propose a method for classifying these data into behavioral annotation, particularly collective behavior of a social group. Our method is based on sequence analysis with a direct encoding of the interactions of a group of wild animals. We evaluate our approach on a real world dataset, showin…
Bialystok University of Technology（比亚韦斯托克理工大学）；Mammal Research Institute, Polish Academy of Sciences（波兰科学院哺乳动物研究所）
Camera traps are used worldwide to monitor wildlife. Despite the increasing availability of Deep Learning (DL) models, the effective usage of this technology to support wildlife monitoring is limited. This is mainly due to the complexity of DL technology and high computing requirements. This paper presents the implementation of the light-weight and state-of-the-art YOLOv5 architecture for automated labeling of camera trap images of mammals in the Bialowieza Forest (BF), Poland. The camera trapping data were organized and harmonized using TRAPPER software, an open source application for managin…
BuckTales : A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes
Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；University of Konstanz（康斯坦茨大学）；Indian Institute of Science（印度科学学院）；BITS Pilani Hyderabad（BITS Pilani海得拉巴校区）
Automatic species classification in camera traps would greatly help the biodiversity monitoring and species analysis in the earth. In order to accelerate the development of automatic species classification task, "Microsoft AI for Earth" have prepared a challenge in FGVC6 workshop at CVPR 2019, which called "iWildCam 2019 competition". In this work, we propose an efficient method for categorizing animals in the wild. We transfer the state-of-the-art ImagaNet pretrained models to the problem. To improve the generalization and robustness of the model, we utilize efficient image augmentation and r…
The ongoing biodiversity crisis calls for accurate estimation of animal density and abundance to identify sources of biodiversity decline and effectiveness of conservation interventions. Camera traps together with abundance estimation methods are often employed for this purpose. The necessary distances between camera and observed animals are traditionally derived in a laborious, fully manual or semi-automatic process. Both approaches require reference image material, which is both difficult to acquire and not available for existing datasets. We propose a fully automatic approach we call AUtoma…
The SA-FARI Dataset: Segment Anything in Footage of Animals for Recognition and Identification
Conservation X Labs (CXL)（Conservation X Labs）；Meta；University of Bristol（布里斯托大学）；Hasso Plattner Institute（哈索罗滕堡研究所）；University of Oviedo（奥维多大学）；Osa Conservation（Osa保护）；Senckenberg Museum of Natural History（Senckenberg自然史博物馆）；Max Planck Institute for Evolutionary Anthropology（马克斯·普朗克进化人类学研究所）；Climate Corridors（气候走廊）
University of Wyoming（怀俄明大学）；Auburn University（奥本大学）；Harvard University（哈佛大学）；University of Oxford（牛津大学）；University of Minnesota（明尼苏达大学）；Uber AI Labs（优步人工智能实验室）
Having accurate, detailed, and up-to-date information about the location and behavior of animals in the wild would revolutionize our ability to study and conserve ecosystems. We investigate the ability to automatically, accurately, and inexpensively collect such data, which could transform many fields of biology, ecology, and zoology into "big data" sciences. Motion sensor "camera traps" enable collecting wildlife pictures inexpensively, unobtrusively, and frequently. However, extracting information from these pictures remains an expensive, time-consuming, manual task. We demonstrate that such…
IMATI-CNR（意大利国家研究委员会信息与远程技术研究所）；University of California, Berkeley（加州大学伯克利分校）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
We present the first method to perform automatic 3D pose, shape and texture capture of animals from images acquired in-the-wild. In particular, we focus on the problem of capturing 3D information about Grevy's zebras from a collection of images. The Grevy's zebra is one of the most endangered species in Africa, with only a few thousand individuals left. Capturing the shape and pose of these animals can provide biologists and conservationists with information about animal health and behavior. In contrast to research on human pose, shape and texture estimation, training data for endangered speci…
Ubisoft China AI & Data Lab（育碧中国AI与数据实验室）；Sichuan University（四川大学）；Ubisoft, Singapore（育碧新加坡）；INRIA Nancy Grand Est（法国国家信息与自动化研究所南锡大东区中心）
This work introduces a novel strategy for generating synthetic training data for 2D and 3D pose estimation of animals using keyframe animations. With the objective to automate the process of creating animations for wildlife, we train several 2D and 3D pose estimation models with synthetic data, and put in place an end-to-end pipeline called ZooBuilder. The pipeline takes as input a video of an animal in the wild, and generates the corresponding 2D and 3D coordinates for each joint of the animal's skeleton. With this approach, we produce motion capture data that can be used to create animations…
University of Udine（乌迪内大学）；University of Edinburgh（爱丁堡大学）；Wildlife Conservation Society（野生生物保护学会）
Photographs of wild animals in their natural habitats can be recorded unobtrusively via cameras that are triggered by motion nearby. The installation of such camera traps is becoming increasingly common across the world. Although this is a convenient source of invaluable data for biologists, ecologists and conservationists, the arduous task of poring through potentially millions of pictures each season introduces prohibitive costs and frustrating delays. We develop automatic algorithms that are able to detect animals, identify the species of animals and to recognize individual animals for two …
Automated animal censuses with aerial imagery are a vital ingredient towards wildlife conservation. Recent models are generally based on deep learning and thus require vast amounts of training data. Due to their scarcity and minuscule size, annotating animals in aerial imagery is a highly tedious process. In this project, we present a methodology to reduce the amount of required training data by resorting to self-supervised pretraining. In detail, we examine a combination of recent contrastive learning methodologies like Momentum Contrast (MoCo) and Cross-Level Instance-Group Discrimination (C…
Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images
Microsoft AI for Good Lab, Redmond（微软人工智能公益实验室）；University of Southern California, Los Angeles（南加州大学）；Microsoft Research, Redmond（微软研究院）；Nanyang Technological University, Singapore（南洋理工大学）；Universidad de Los Andes, Colombia（洛斯安第斯大学）
Pytorch-Wildlife: A Collaborative Deep Learning Framework for Conservation
Microsoft AI for Good（微软AI公益部门）；Center for Research and Formation in Artificial Intelligence, Universidad de los Andes（安第斯大学人工智能研究与形成中心）；Massachusetts Institute of Technology（麻省理工学院）
Automated animal censuses with aerial imagery are a vital ingredient towards wildlife conservation. Recent models are generally based on supervised learning and thus require vast amounts of training data. Due to their scarcity and minuscule size, annotating animals in aerial imagery is a highly tedious process. In this project, we present a methodology to reduce the amount of required training data by resorting to self-supervised pretraining. In detail, we examine a combination of recent contrastive learning methodologies like Momentum Contrast (MoCo) and Cross-Level Instance-Group Discriminat…
Creating high-quality articulated 3D models of animals is challenging either via manual creation or using 3D scanning tools. Therefore, techniques to reconstruct articulated 3D objects from 2D images are crucial and highly useful. In this work, we propose a practical problem setting to estimate 3D pose and shape of animals given only a few (10-30) in-the-wild images of a particular animal species (say, horse). Contrary to existing works that rely on pre-defined template shapes, we do not assume any form of 2D or 3D ground-truth annotations, nor do we leverage any multi-view or temporal informa…
Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；Centre for the Advanced Study of Collective Behaviour, University of Konstanz（康斯坦茨大学集体行为高级研究中心）；University of Konstanz（康斯坦茨大学）；Institute for Applied Mathematics and Information Technologies, IMATI-CNR（意大利国家研究委员会应用数学与信息技术研究所）；University of Münster（明斯特大学）；Ecole Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Wageningen University（瓦赫宁根大学）；University of Bristol（布里斯托尔大学）；North Carolina State University（北卡罗来纳州立大学）；North Carolina Museum of Natural Sciences（北卡罗来纳自然科学博物馆）；Cornell Lab of Ornithology, Cornell University（康奈尔大学鸟类学实验室）；Cornell University（康奈尔大学）
Data acquisition in animal ecology is rapidly accelerating due to inexpensive and accessible sensors such as smartphones, drones, satellites, audio recorders and bio-logging devices. These new technologies and the data they generate hold great potential for large-scale environmental monitoring and understanding, but are limited by current data processing approaches which are inefficient in how they ingest, digest, and distill data into relevant information. We argue that machine learning, and especially deep learning approaches, can meet this analytic challenge to enhance our understanding, mo…
Identifying animals from a large group of possible individuals is very important for biodiversity monitoring and especially for collecting data on a small number of particularly interesting individuals, as these have to be identified first before this can be done. Identifying them can be a very time-consuming task. This is especially true, if the animals look very similar and have only a small number of distinctive features, like elephants do. In most cases the animals stay at one place only for a short period of time during which the animal needs to be identified for knowing whether it is imp…
Calibrated Similarity and Graph Clustering for Open-Set Animal Re-Identification
Made In Alexandria Artificial Intelligence Team（亚历山大制造人工智能团队）；Faculty of Computer Science and Engineering, Alamein International University（阿拉曼国际大学计算机科学与工程学院）；Faculty of Computers and Data Science, Alexandria University（亚历山大大学计算机与数据科学学院）；Faculty of Engineering, Alexandria University（亚历山大大学工程学院）；Alexandria Higher Institute of Engineering and Technology（亚历山大高等工程与技术学院）；Moscow Pedagogical State University (MPGU University)（莫斯科国立师范大学（MPGU大学））
