Camera traps are important tools in animal ecology for biodiversity monitoring and conservation. However, their practical application is limited by issues such as poor generalization to new and unseen locations. Images are typically associated with diverse forms of context, which may exist in different modalities. In this work, we exploit the structured context linked to camera trap images to boost out-of-distribution generalization for species classification tasks in camera traps. For instance, a picture of a wild animal could be linked to details about the time and place it was captured, as well as structured biological knowledge about the animal species. While often overlooked by existing studies, incorporating such context offers several potential benefits for better image understanding, such as addressing data scarcity and enhancing generalization. However, effectively incorporating such heterogeneous context into the visual domain is a challenging problem. To address this, we propose a novel framework that transforms species classification as link prediction in a multimodal knowledge graph (KG). This framework enables the seamless integration of diverse multimodal contexts for visual recognition. We apply this framework for out-of-distribution species classification on the iWildCam2020-WILDS and Snapshot Mountain Zebra datasets and achieve competitive performance with state-of-the-art approaches. Furthermore, our framework enhances sample efficiency for recognizing under-represented species.
Universidad de Antioquia UdeA（安蒂奥基亚大学（UdeA））；Instituto Tecnológico Metropolitano ITM（大都会理工学院（ITM））
Non intrusive monitoring of animals in the wild is possible using camera trapping framework, which uses cameras triggered by sensors to take a burst of images of animals in their habitat. However camera trapping framework produces a high volume of data (in the order on thousands or millions of images), which must be analyzed by a human expert. In this work, a method for animal species identification in the wild using very deep convolutional neural networks is presented. Multiple versions of the Snapshot Serengeti dataset were used in order to probe the ability of the method to cope with differ…
CamAgent: An LLM-Agent Framework for Multi-Species Camera-Trap Workflows
Beijing Normal University（北京师范大学）；National Forestry and Grassland Administration Key Laboratory for Conservation Ecology of Northeast Tiger and Leopard（国家林业和草原局东北虎豹保护生态学重点实验室）
Towards Context-Rich Automated Biodiversity Assessments: Deriving AI-Powered Insights from Camera Trap Data
Liverpool John Moores University（利物浦约翰摩尔斯大学）；Chester Zoo（切斯特动物园）；Welgevonden Game Reserve（韦尔赫翁登野生动物保护区）；University of St Andrews（圣安德鲁斯大学）；University of Aberdeen（阿伯丁大学）
California Institute of Technology（加州理工学院）；Google（谷歌）
Camera traps enable the automatic collection of large quantities of image data. Ecologists use camera traps to monitor animal populations all over the world. In order to estimate the abundance of a species from camera trap data, ecologists need to know not just which species were seen, but also how many individuals of each species were seen. Object detection techniques can be used to find the number of individuals in each image. However, since camera traps collect images in motion-triggered bursts, simply adding up the number of detections over all frames is likely to lead to an incorrect esti…
Bialystok University of Technology（比亚韦斯托克理工大学）；Mammal Research Institute, Polish Academy of Sciences（波兰科学院哺乳动物研究所）
Camera traps are used worldwide to monitor wildlife. Despite the increasing availability of Deep Learning (DL) models, the effective usage of this technology to support wildlife monitoring is limited. This is mainly due to the complexity of DL technology and high computing requirements. This paper presents the implementation of the light-weight and state-of-the-art YOLOv5 architecture for automated labeling of camera trap images of mammals in the Bialowieza Forest (BF), Poland. The camera trapping data were organized and harmonized using TRAPPER software, an open source application for managin…
Trail camera imagery has increasingly gained popularity amongst biologists for conservation and ecological research. Minimal human interference required to operate camera traps allows capturing unbiased species activities. Several studies - based on human and wildlife interactions, migratory patterns of various species, risk of extinction in endangered populations - are limited by the lack of rich data and the time-consuming nature of manually annotating trail camera imagery. We introduce a challenging wildlife camera trap classification dataset collected from two different locations in Southw…
Empowering Wildlife Guardians: An Equitable Digital Stewardship and Reward System for Biodiversity Conservation using Deep Learning and 3/4G Camera Traps
Liverpool John Moores University（利物浦约翰摩尔斯大学）；Gap Africa Projects（Gap非洲项目）；Welgevonden Game Reserve（韦尔格翁登禁猎区）；Borneo Futures（婆罗洲未来组织）；Czech Technical University（捷克技术大学）
Camera traps are a method for monitoring wildlife and they collect a large number of pictures. The number of images collected of each species usually follows a long-tail distribution, i.e., a few classes have a large number of instances, while a lot of species have just a small percentage. Although in most cases these rare species are the ones of interest to ecologists, they are often neglected when using deep-learning models because these models require a large number of images for the training. In this work, a simple and effective framework called Square-Root Sampling Branch (SSB) is propose…
Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images
Microsoft AI for Good Lab, Redmond（微软人工智能公益实验室）；University of Southern California, Los Angeles（南加州大学）；Microsoft Research, Redmond（微软研究院）；Nanyang Technological University, Singapore（南洋理工大学）；Universidad de Los Andes, Colombia（洛斯安第斯大学）
Camera Traps (or Wild Cams) enable the automatic collection of large quantities of image data. Biologists all over the world use camera traps to monitor biodiversity and population density of animal species. The computer vision community has been making strides towards automating the species classification challenge in camera traps, but as we try to expand the scope of these models from specific regions where we have collected training data to different areas we are faced with an interesting problem: how do you classify a species in a new region that you may not have seen in previous training …
Deep learning-based ecological analysis of camera trap images is impacted by training data quality and quantity
University College London（伦敦大学学院）；Zoological Society of London（伦敦动物学会）；Instituto Biotrópicos（Biotrópicos研究所）；University of Kent（肯特大学）；University of Oxford（牛津大学）；Tribhuvan University（特里布万大学）；University of Edinburgh（爱丁堡大学）
California Institute of Technology（加州理工学院）；Google（谷歌公司）
Camera traps enable the automatic collection of large quantities of image data. Biologists all over the world use camera traps to monitor animal populations. We have recently been making strides towards automatic species classification in camera trap images. However, as we try to expand the geographic scope of these models we are faced with an interesting question: how do we train models that perform well on new (unseen during training) camera trap locations? Can we leverage data from other modalities, such as citizen science data and remote sensing data? In order to tackle this problem, we ha…
California Institute of Technology（加州理工学院）；Microsoft AI for Earth（微软地球人工智能）
Biologists all over the world use camera traps to monitor biodiversity and wildlife population density. The computer vision community has been making strides towards automating the species classification challenge in camera traps, but it has proven difficult to to apply models trained in one region to images collected in different geographic areas. In some cases, accuracy falls off catastrophically in new region, due to both changes in background and the presence of previously-unseen species. We propose a pipeline that takes advantage of a pre-trained general animal detector and a smaller set …
Visual animal biometrics is rapidly gaining popularity as it enables a non-invasive and cost-effective approach for wildlife monitoring applications. Widespread usage of camera traps has led to large volumes of collected images, making manual processing of visual content hard to manage. In this work, we develop a framework for automatic detection and recognition of individuals in different patterned species like tigers, zebras and jaguars. Most existing systems primarily rely on manual input for localizing the animal, which does not scale well to large datasets. In order to automate the detect…
Microsoft AI for Earth（微软地球人工智能项目）；University of Wyoming（怀俄明大学）；California Institute of Technology（加州理工学院）；Microsoft Research（微软研究院）；Uber AI（优步人工智能）
Biodiversity conservation depends on accurate, up-to-date information about wildlife population distributions. Motion-activated cameras, also known as camera traps, are a critical tool for population surveys, as they are cheap and non-intrusive. However, extracting useful information from camera trap images is a cumbersome process: a typical camera trap survey may produce millions of images that require slow, expensive manual review. Consequently, critical information is often lost due to resource limitations, and critical conservation questions may be answered too slowly to support decision-m…
University of Bonn（波恩大学）；German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig（德国哈勒-耶拿-莱比锡综合生物多样性研究中心（iDiv））
The development and application of modern technology is an essential basis for the efficient monitoring of species in natural habitats and landscapes to trace the development of ecosystems, species communities, and populations, and to analyze reasons of changes. For estimating animal abundance using methods such as camera trap distance sampling, spatial information of natural habitats in terms of 3D (three-dimensional) measurements is crucial. Additionally, 3D information improves the accuracy of animal detection using camera trapping. This study presents a novel approach to 3D camera trapping…
