相机陷阱已成为野生动物保护的重要工具，提供了非侵入性的手段来监测和研究自然栖息地中的野生动物。利用目标检测算法自动识别相机陷阱图像中的物种对研究和保护目的具有重大意义。然而，泛化问题普遍存在，即训练好的模型无法将其学到的知识应用于从未见过的数据集。本论文探讨了对YOLOv8目标检测算法的改进，以解决泛化问题。本研究深入分析了基线YOLOv8模型的局限性，强调了其在真实世界环境中泛化能力不足的困境。为克服这些局限性，提出了多项改进，包括引入全局注意力机制（GAM）模块、改进的多尺度特征融合，以及将Wise Intersection over Union（WIoUv3）作为边界框回归损失函数。全面的评估和消融实验表明，改进后的模型能够抑制背景噪声，聚焦于目标属性，并在新环境中展现出稳健的泛化能力。所提出的改进不仅解决了相机陷阱数据集固有的挑战，还为在真实世界保护场景中的更广泛应用铺平了道路，最终有助于有效管理野生动物种群及其栖息地。
英文摘要：
Camera traps have become integral tools in wildlife conservation, providing non-intrusive means to monitor and study wildlife in their natural habitats. The utilization of object detection algorithms to automate species identification from Camera Trap images is of huge importance for research and conservation purposes. However, the generalization issue, where the trained model is unable to apply its learnings to a never-before-seen dataset, is prevalent. This thesis explores the enhancements made to the YOLOv8 object detection algorithm to address the problem of generalization. The study delves into the limitations of the baseline YOLOv8 model, emphasizing its struggles with generalization in real-world environments. To overcome these limitations, enhancements are proposed, including the incorporation of a Global Attention Mechanism (GAM) module, modified multi-scale feature fusion, and Wise Intersection over Union (WIoUv3) as a bounding box regression loss function. A thorough evaluation and ablation experiments reveal the improved model's ability to suppress the background noise, focus on object properties, and exhibit robust generalization in novel environments. The proposed enhancements not only address the challenges inherent in camera trap datasets but also pave the way for broader applicability in real-world conservation scenarios, ultimately aiding in the effective management of wildlife populations and habitats.
Bialystok University of Technology（比亚韦斯托克理工大学）；Mammal Research Institute, Polish Academy of Sciences（波兰科学院哺乳动物研究所）
Camera traps are used worldwide to monitor wildlife. Despite the increasing availability of Deep Learning (DL) models, the effective usage of this technology to support wildlife monitoring is limited. This is mainly due to the complexity of DL technology and high computing requirements. This paper presents the implementation of the light-weight and state-of-the-art YOLOv5 architecture for automated labeling of camera trap images of mammals in the Bialowieza Forest (BF), Poland. The camera trapping data were organized and harmonized using TRAPPER software, an open source application for managin…
University of Guelph（圭尔夫大学）；Vector Institute for Artificial Intelligence（向量人工智能研究所）；Canadian Institute for Advanced Research（加拿大高级研究所）
Deep learning methods for computer vision tasks show promise for automating the data analysis of camera trap images. Ecological camera traps are a common approach for monitoring an ecosystem's animal population, as they provide continual insight into an environment without being intrusive. However, the analysis of camera trap images is expensive, labour intensive, and time consuming. Recent advances in the field of deep learning for object detection show promise towards automating the analysis of camera trap images. Here, we demonstrate their capabilities by training and comparing two deep lea…
TotalEnergies SE（道达尔能源公司）；University of Lille（里尔大学）；CNRS（法国国家科学研究中心）；MODIS
Camera traps have revolutionized the animal research of many species that were previously nearly impossible to observe due to their habitat or behavior. They are cameras generally fixed to a tree that take a short sequence of images when triggered. Deep learning has the potential to overcome the workload to automate image classification according to taxon or empty images. However, a standard deep neural network classifier fails because animals often represent a small portion of the high-definition images. That is why we propose a workflow named Weakly Object Detection Faster-RCNN+FPN which sui…
Ove Arup and Partners（奥雅纳工程顾问公司）；Newcastle University（纽卡斯尔大学）
Understanding the abundance of a species is the first step towards understanding both its long-term sustainability and the impact that we may be having upon it. Ecologists use camera traps to remotely survey for the presence of specific animal species. Previous studies have shown that deep learning models can be trained to automatically detect and classify animals within camera trap imagery with high levels of confidence. However, the ability to train these models is reliant upon having enough high-quality training data. What happens when the animal is rare or the data sets are non-existent? T…
We propose the first multi-frame video object detection framework trained to detect great apes. It is applicable to challenging camera trap footage in complex jungle environments and extends a traditional feature pyramid architecture by adding self-attention driven feature blending in both the spatial as well as the temporal domain. We demonstrate that this extension can detect distinctive species appearance and motion signatures despite significant partial occlusion. We evaluate the framework using 500 camera trap videos of great apes from the Pan African Programme containing 180K frames, whi…
Camera traps are a proven tool in biology and specifically biodiversity research. However, camera traps including depth estimation are not widely deployed, despite providing valuable context about the scene and facilitating the automation of previously laborious manual ecological methods. In this study, we propose an automated camera trap-based approach to detect and identify animals using depth estimation. To detect and identify individual animals, we propose a novel method D-Mask R-CNN for the so-called instance segmentation which is a deep learning-based technique to detect and delineate ea…
Monitoring wildlife through camera traps produces a massive amount of images, whose a significant portion does not contain animals, being later discarded. Embedding deep learning models to identify animals and filter these images directly in those devices brings advantages such as savings in the storage and transmission of data, usually resource-constrained in this type of equipment. In this work, we present a comparative study on animal recognition models to analyze the trade-off between precision and inference latency on edge devices. To accomplish this objective, we investigate classifiers …
Democratising Camera Trap AI: An Open-Source Model for Detecting UK Mammals
Liverpool John Moores University（利物浦约翰穆里斯大学）；Durham University（杜伦大学）；MammalWeb（哺乳动物网）；Game & Wildlife Conservation Trust（游戏与野生动物保护信托）；National Trust（国家信托）；Animal and Plant Health Agency（动物和植物卫生局）；Chester Zoo（切斯特动物园）；University of St Andrews（圣安德鲁大学）；Nottingham Trent University（诺丁汉特伦特大学）
Improving 2D Human Pose Estimation in Rare Camera Views with Synthetic Data
Czech Technical University in Prague（布拉格捷克技术大学）；Faculty of Electrical Engineering（电气工程学院）；Department of Cybernetics（控制论系）；Visual Recognition Group（视觉识别组）
California Institute of Technology（加州理工学院）；Google（谷歌公司）
In static monitoring cameras, useful contextual information can stretch far beyond the few seconds typical video understanding models might see: subjects may exhibit similar behavior over multiple days, and background objects remain static. Due to power and storage constraints, sampling frequencies are low, often no faster than one frame per second, and sometimes are irregular due to the use of a motion trigger. In order to perform well in this setting, models must be robust to irregular sampling rates. In this paper we propose a method that leverages temporal context from the unlabeled frames…
Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）；Google（谷歌）
We leverage unsupervised learning of depth, egomotion, and camera intrinsics to improve the performance of single-image semantic segmentation, by enforcing 3D-geometric and temporal consistency of segmentation masks across video frames. The predicted depth, egomotion, and camera intrinsics are used to provide an additional supervision signal to the segmentation model, significantly enhancing its quality, or, alternatively, reducing the number of labels the segmentation model needs. Our experiments were performed on the ScanNet dataset.
University of Bonn（波恩大学）；German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig（德国哈勒-耶拿-莱比锡综合生物多样性研究中心（iDiv））
The development and application of modern technology is an essential basis for the efficient monitoring of species in natural habitats and landscapes to trace the development of ecosystems, species communities, and populations, and to analyze reasons of changes. For estimating animal abundance using methods such as camera trap distance sampling, spatial information of natural habitats in terms of 3D (three-dimensional) measurements is crucial. Additionally, 3D information improves the accuracy of animal detection using camera trapping. This study presents a novel approach to 3D camera trapping…
Visual animal biometrics is rapidly gaining popularity as it enables a non-invasive and cost-effective approach for wildlife monitoring applications. Widespread usage of camera traps has led to large volumes of collected images, making manual processing of visual content hard to manage. In this work, we develop a framework for automatic detection and recognition of individuals in different patterned species like tigers, zebras and jaguars. Most existing systems primarily rely on manual input for localizing the animal, which does not scale well to large datasets. In order to automate the detect…
A new open-source image processing pipeline for analyzing camera trap time-lapse recordings is described. This pipeline includes machine learning models to assist human-in-the-loop video segmentation and animal re-identification. We present some performance results and observations on the utility of this pipeline after using it in a year-long project studying the spatial ecology and social behavior of the gopher tortoise.
University of Washington（华盛顿大学）；Microsoft AI for Earth（微软地球人工智能项目）
Camera Traps are extensively used to observe wildlife in their natural habitat without disturbing the ecosystem. This could help in the early detection of natural or human threats to animals, and help towards ecological conservation. Currently, a massive number of such camera traps have been deployed at various ecological conservation areas around the world, collecting data for decades, thereby requiring automation to detect images containing animals. Existing systems perform classification to detect if images contain animals by considering a single image. However, due to challenging scenes wi…
A machine learning pipeline for automated insect monitoring
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；Federal University of Amazonas（亚马逊联邦大学）；EPFL（洛桑联邦理工学院）；Daresay（达雷赛公司）；Montreal Insectarium（蒙特利尔昆虫馆）；McGill University（麦吉尔大学）
The estimation of the camera poses associated with a set of images commonly relies on feature matches between the images. In contrast, we are the first to address this challenge by using objectness regions to guide the pose estimation problem rather than explicit semantic object detections. We propose Pose Refiner Network (PoserNet) a light-weight Graph Neural Network to refine the approximate pair-wise relative camera poses. PoserNet exploits associations between the objectness regions - concisely expressed as bounding boxes - across multiple views to globally refine sparsely connected view g…
By leveraging deep learning to automatically classify camera trap images, ecologists can monitor biodiversity conservation efforts and the effects of climate change on ecosystems more efficiently. Due to the imbalanced class-distribution of camera trap datasets, current models are biased towards the majority classes. As a result, they obtain good performance for a few majority classes but poor performance for many minority classes. We used two-phase training to increase the performance for these minority classes. We trained, next to a baseline model, four models that implemented a different ve…
Universidad de Antioquia UdeA（安蒂奥基亚大学（UdeA））；Instituto Tecnológico Metropolitano ITM（大都会理工学院（ITM））
Non intrusive monitoring of animals in the wild is possible using camera trapping framework, which uses cameras triggered by sensors to take a burst of images of animals in their habitat. However camera trapping framework produces a high volume of data (in the order on thousands or millions of images), which must be analyzed by a human expert. In this work, a method for animal species identification in the wild using very deep convolutional neural networks is presented. Multiple versions of the Snapshot Serengeti dataset were used in order to probe the ability of the method to cope with differ…
Multimodal Foundation Models for Zero-shot Animal Species Recognition in Camera Trap Images
Microsoft AI for Good Lab, Redmond（微软人工智能公益实验室）；University of Southern California, Los Angeles（南加州大学）；Microsoft Research, Redmond（微软研究院）；Nanyang Technological University, Singapore（南洋理工大学）；Universidad de Los Andes, Colombia（洛斯安第斯大学）
The segmentation and classification of animals from camera-trap images is due to the conditions under which the images are taken, a difficult task. This work presents a method for classifying and segmenting mammal genera from camera-trap images. Our method uses Multi-Layer Robust Principal Component Analysis (RPCA) for segmenting, Convolutional Neural Networks (CNNs) for extracting features, Least Absolute Shrinkage and Selection Operator (LASSO) for selecting features, and Artificial Neural Networks (ANNs) or Support Vector Machines (SVM) for classifying mammal genera present in the Colombian…
Label distributions in camera-trap images are highly imbalanced and long-tailed, resulting in neural networks tending to be biased towards head-classes that appear frequently. Although long-tail learning has been extremely explored to address data imbalances, few studies have been conducted to consider camera-trap characteristics, such as multi-domain and multi-frame setup. Here, we propose a unified framework and introduce two datasets for long-tailed camera-trap recognition. We first design domain experts, where each expert learns to balance imperfect decision boundaries caused by data imbal…
Microsoft AI for Earth（微软地球人工智能项目）；University of Wyoming（怀俄明大学）；California Institute of Technology（加州理工学院）；Microsoft Research（微软研究院）；Uber AI（优步人工智能）
Biodiversity conservation depends on accurate, up-to-date information about wildlife population distributions. Motion-activated cameras, also known as camera traps, are a critical tool for population surveys, as they are cheap and non-intrusive. However, extracting useful information from camera trap images is a cumbersome process: a typical camera trap survey may produce millions of images that require slow, expensive manual review. Consequently, critical information is often lost due to resource limitations, and critical conservation questions may be answered too slowly to support decision-m…
California Institute of Technology（加州理工学院）；Microsoft AI for Earth（微软地球人工智能）
Biologists all over the world use camera traps to monitor biodiversity and wildlife population density. The computer vision community has been making strides towards automating the species classification challenge in camera traps, but it has proven difficult to to apply models trained in one region to images collected in different geographic areas. In some cases, accuracy falls off catastrophically in new region, due to both changes in background and the presence of previously-unseen species. We propose a pipeline that takes advantage of a pre-trained general animal detector and a smaller set …
California Institute of Technology（加州理工学院）；Harvard University（哈佛大学）
To protect the location of camera trap data containing sensitive, high-target species, many ecologists randomly obfuscate the latitude and longitude of the camera when publishing their data. For example, they may publish a random location within a 1km radius of the true camera location for each camera in their network. In this paper, we investigate the robustness of geo-obfuscation for maintaining camera trap location privacy, and show via a case study that a few simple, intuitive heuristics and publicly available satellite rasters can be used to reduce the area likely to contain the camera by…
University of Guelph（圭尔夫大学）；Vector Institute for Artificial Intelligence（向量人工智能研究所）
The ability of a researcher to re-identify (re-ID) an individual animal upon re-encounter is fundamental for addressing a broad range of questions in the study of ecosystem function, community and population dynamics, and behavioural ecology. In this review, we describe a brief history of camera traps for re-ID, present a collection of computer vision feature engineering methodologies previously used for animal re-ID, provide an introduction to the underlying mechanisms of deep learning relevant to animal re-ID, highlight the success of deep learning methods for human re-ID, describe the few e…
The segmentation of animals from camera-trap images is a difficult task. To illustrate, there are various challenges due to environmental conditions and hardware limitation in these images. We proposed a multi-layer robust principal component analysis (multi-layer RPCA) approach for background subtraction. Our method computes sparse and low-rank images from a weighted sum of descriptors, using color and texture features as case of study for camera-trap images segmentation. The segmentation algorithm is composed of histogram equalization or Gaussian filtering as pre-processing, and morphologica…
California Institute of Technology（加州理工学院）；Google（谷歌公司）
Camera traps enable the automatic collection of large quantities of image data. Biologists all over the world use camera traps to monitor animal populations. We have recently been making strides towards automatic species classification in camera trap images. However, as we try to expand the geographic scope of these models we are faced with an interesting question: how do we train models that perform well on new (unseen during training) camera trap locations? Can we leverage data from other modalities, such as citizen science data and remote sensing data? In order to tackle this problem, we ha…
Trail camera imagery has increasingly gained popularity amongst biologists for conservation and ecological research. Minimal human interference required to operate camera traps allows capturing unbiased species activities. Several studies - based on human and wildlife interactions, migratory patterns of various species, risk of extinction in endangered populations - are limited by the lack of rich data and the time-consuming nature of manually annotating trail camera imagery. We introduce a challenging wildlife camera trap classification dataset collected from two different locations in Southw…
University of Wyoming（怀俄明大学）；Auburn University（奥本大学）；Harvard University（哈佛大学）；University of Oxford（牛津大学）；University of Minnesota（明尼苏达大学）；Uber AI Labs（优步人工智能实验室）
Having accurate, detailed, and up-to-date information about the location and behavior of animals in the wild would revolutionize our ability to study and conserve ecosystems. We investigate the ability to automatically, accurately, and inexpensively collect such data, which could transform many fields of biology, ecology, and zoology into "big data" sciences. Motion sensor "camera traps" enable collecting wildlife pictures inexpensively, unobtrusively, and frequently. However, extracting information from these pictures remains an expensive, time-consuming, manual task. We demonstrate that such…
CamAgent: An LLM-Agent Framework for Multi-Species Camera-Trap Workflows
Beijing Normal University（北京师范大学）；National Forestry and Grassland Administration Key Laboratory for Conservation Ecology of Northeast Tiger and Leopard（国家林业和草原局东北虎豹保护生态学重点实验室）
University of Bonn（波恩大学）；German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig（德国综合生物多样性研究中心（iDiv）哈勒-耶拿-莱比锡）
The biodiversity crisis is still accelerating, despite increasing efforts by the international community. Estimating animal abundance is of critical importance to assess, for example, the consequences of land-use change and invasive species on community composition, or the effectiveness of conservation interventions. Various approaches have been developed to estimate abundance of unmarked animal populations. Whereas these approaches differ in methodological details, they all require the estimation of the effective area surveyed in front of a camera trap. Until now camera-to-animal distance mea…
The ongoing biodiversity crisis calls for accurate estimation of animal density and abundance to identify sources of biodiversity decline and effectiveness of conservation interventions. Camera traps together with abundance estimation methods are often employed for this purpose. The necessary distances between camera and observed animals are traditionally derived in a laborious, fully manual or semi-automatic process. Both approaches require reference image material, which is both difficult to acquire and not available for existing datasets. We propose a fully automatic approach we call AUtoma…
Improving Robustness of LiDAR-Camera Fusion Model against Weather Corruption from Fusion Strategy Perspective
Nanyang Technological University（南洋理工大学）；East China Normal University（华东师范大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；New York University（纽约大学）
California Institute of Technology（加州理工学院）；Google（谷歌）
Camera traps enable the automatic collection of large quantities of image data. Ecologists use camera traps to monitor animal populations all over the world. In order to estimate the abundance of a species from camera trap data, ecologists need to know not just which species were seen, but also how many individuals of each species were seen. Object detection techniques can be used to find the number of individuals in each image. However, since camera traps collect images in motion-triggered bursts, simply adding up the number of detections over all frames is likely to lead to an incorrect esti…
Tracking Phenological Status and Ecological Interactions in a Hawaiian Cloud Forest Understory using Low-Cost Camera Traps and Visual Foundation Models
Computer Science, The Ohio State University, Columbus, Ohio, USA（俄亥俄州立大学计算机科学系）；Computer Science, The University of Puerto Rico Rio Piedras, San Juan, Puerto Rico（波多黎各大学里奥皮埃拉斯分校计算机科学系）；Computer Science, McGill University, Montreal, Quebec, Canada（麦吉尔大学计算机科学系）；Battele Ecology, Neon Domain 20, Hilo, Hawaii（Battele生态研究所）；Ecology, Princeton University, Princeton, New Jersey, USA（普林斯顿大学生态学系）
This paper work directly towards the improving the quality of the image for the digital cameras and other visual capturing products. In this Paper, the authors clearly defines the problems occurs in the CFA image. A different methodology for removing the noise is discuses in the paper for color correction and color balancing of the image. At the same time, the authors also proposed a new methodology of providing denoisiing process before the demosaickingfor the improving the image quality of CFA which is much efficient then the other previous defined. The demosaicking process for producing the…
Camera traps, unmanned observation devices, and deep learning-based image recognition systems have greatly reduced human effort in collecting and analyzing wildlife images. However, data collected via above apparatus exhibits 1) long-tailed and 2) open-ended distribution problems. To tackle the open-set long-tailed recognition problem, we propose the Temporal Flow Mask Attention Network that comprises three key building blocks: 1) an optical flow module, 2) an attention residual module, and 3) a meta-embedding classifier. We extract temporal features of sequential frames using the optical flow…
Towards Scalable Insect Monitoring: Ultra-Lightweight CNNs as On-Device Triggers for Insect Camera Traps
University of Exeter（埃克塞特大学）；Institute for Data Science and AI, University of Exeter（埃克塞特大学数据科学与人工智能学院）；Centre for Ecology and Conservation, University of Exeter（埃克塞特大学生态与保护中心）
针对 PIR 传感器难以触发昆虫相机陷阱的问题，本文提出在低功耗微控制器上运行超轻量级 CNN 检测连续图像中的昆虫，实现零延迟、高 AUC、低功耗与更长部署。
Spotted: Location-informed Reidentification of Hyenas and Leopards in Camera Trap Surveys
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）；Department of Computer Science, University of Freiburg（弗莱堡大学计算机科学系）；Bubye Valley Conservancy（布比耶谷保护区）；Wildlife Conservation Research Unit, Department of Biology, University of Oxford（牛津大学生物系野生动物保护研究组）；Gonarezhou National Park（戈纳雷若国家公园）；Oxford e-Research Centre, University of Oxford（牛津大学牛津电子研究中心）
Monocular depth predictors are typically trained on large-scale training sets which are naturally biased w.r.t the distribution of camera poses. As a result, trained predictors fail to make reliable depth predictions for testing examples captured under uncommon camera poses. To address this issue, we propose two novel techniques that exploit the camera pose during training and prediction. First, we introduce a simple perspective-aware data augmentation that synthesizes new training examples with more diverse views by perturbing the existing ones in a geometrically consistent manner. Second, we…
