PanAf20K: A Large Video Dataset for Wild Ape Detection and Behaviour Recognition
University of Bristol（布里斯托尔大学）；Max Planck Institute for Evolutionary Anthropology（马克斯·普朗克进化人类学研究所）；Wild Chimpanzee Foundation（野生黑猩猩基金会）；University of Warsaw（华沙大学）
Otto Brookes, Majid Mirmehdi, Colleen Stephens, Samuel Angedakin, Katherine Corogenes, Dervla Dowd, Paula Dieguez, Thurston C. Hicks, Sorrel Jones, Kevin Lee, V… 展开作者
Otto Brookes, Majid Mirmehdi, Colleen Stephens, Samuel Angedakin, Katherine Corogenes, Dervla Dowd, Paula Dieguez, Thurston C. Hicks, Sorrel Jones, Kevin Lee, Vera Leinert, Juan Lapuente, Maureen S. McCarthy, Amelia Meier, Mizuki Murai, Emmanuelle Normand, Virginie Vergnes, Erin G. Wessling, Roman M. Wittig, Kevin Langergraber, Nuria Maldonado, Xinyu Yang, Klaus Zuberbuhler, Christophe Boesch, Mimi Arandjelovic, Hjalmar Kuhl, Tilo Burghardt
我们提出了PanAf20K数据集，这是目前规模最大、多样性最丰富的开放获取带标注野生大猿自然环境视频数据集。该数据集包含超过700万帧画面，来自约20000段黑猩猩与大猩猩的相机陷阱视频，这些视频采集于非洲热带地区的14个野外站点，是“泛非计划：有文化的黑猩猩（Pan African Programme: The Cultured Chimpanzee）”的组成部分。视频配有丰富的标注集与基准测试集，适用于训练和测试多种具有挑战性且具备生态重要性的计算机视觉任务，包括猿类检测与行为识别。鉴于国际自然保护联盟（International Union for Conservation of Nature）目前已将大猿科的所有物种列为濒危或极危物种，推进相机陷阱信息的AI分析至关重要。我们希望该数据集能为AI界的参与提供坚实基础，以提升性能、效率与结果解读能力，从而支持对大猿存在情况、种群数量、分布与行为的评估，助力保护工作。
英文摘要：
We present the PanAf20K dataset, the largest and most diverse open-access annotated video dataset of great apes in their natural environment. It comprises more than 7 million frames across ~20,000 camera trap videos of chimpanzees and gorillas collected at 14 field sites in tropical Africa as part of the Pan African Programme: The Cultured Chimpanzee. The footage is accompanied by a rich set of annotations and benchmarks making it suitable for training and testing a variety of challenging and ecologically important computer vision tasks including ape detection and behaviour recognition. Furthering AI analysis of camera trap information is critical given the International Union for Conservation of Nature now lists all species in the great ape family as either Endangered or Critically Endangered. We hope the dataset can form a solid basis for engagement of the AI community to improve performance, efficiency, and result interpretation in order to support assessments of great ape presence, abundance, distribution, and behaviour and thereby aid conservation efforts.
The PanAf-SBR Dataset: Social Behaviour Recognition for Wild Great Apes
University of Bristol（布里斯托大学）；Wild Chimpanzee Foundation（野生黑猩猩基金会）；Carnegie Mellon University（卡内基梅隆大学）；University of California（加利福尼亚大学）；Peking University（北京大学）；Max Planck Institute for Evolutionary Anthropology（马克斯·普朗克进化人类学研究所）；Senckenberg Museum of Natural History（森肯伯格自然历史博物馆）
In recent years, transformer architecture has been a dominating paradigm in many applications, including affective computing. In this report, we propose our transformer-based model to handle Emotion Classification Task in the 5th Affective Behavior Analysis In-the-wild Competition. By leveraging the attentive model and the synthetic dataset, we attain a score of 0.4775 on the validation set of Aff-Wild2, the dataset provided by the organizer.
Light field (LF) cameras provide rich spatio-angular visual representations by sensing the visual scene from multiple perspectives and have recently emerged as a promising technology to boost the performance of human-machine systems such as biometrics and affective computing. Despite the significant success of LF representation for constrained facial image analysis, this technology has never been used for face and expression recognition in the wild. In this context, this paper proposes a new deep face and expression recognition solution, called CapsField, based on a convolutional neural networ…
The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition
University of Bristol（布里斯托尔大学）；Wild Chimpanzee Foundation（野生黑猩猩基金会）；Martin Luther University Halle-Wittenberg（马丁路德大学哈雷-维滕贝格分校）；Leipzig University（莱比锡大学）；Max Planck Institute for Evolutionary Anthropology（马克斯·普朗克进化人类学研究所）；University of Warsaw（华沙大学）；Harvard University（哈佛大学）；University of Lyon（里昂大学）；Arizona State University（亚利桑那州立大学）；University of St Andrews（圣安德鲁斯大学）；Lancaster University in Leipzig（莱比锡兰卡斯特大学）；Senckenberg Museum of Natural History（森肯伯格自然历史博物馆）
We propose a first great ape-specific visual behaviour recognition system utilising deep learning that is capable of detecting nine core ape behaviours.
Hospital Clínic Barcelona, Universitat de Barcelona, IDIBAPS（巴塞罗那临床医院，巴塞罗那大学，IDIBAPS）；IBM Research AI, T J Watson Research Center（IBM研究AI，T J Watson研究中心）；Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center（医学部皮肤科，纪念斯隆-凯特琳癌症中心）；Kitware；Signal Theory and Communications, Universitat Politècnica de Catalunya（信号理论与通信，加泰罗尼亚理工大学）；Melanoma Unit, Dermatology Department, Hospital Clínic Barcelona, Universitat de Barcelona, IDIBAPS（黑素瘤科，皮肤科，巴塞罗那临床医院，巴塞罗那大学，IDIBAPS）
This article summarizes the BCN20000 dataset, composed of 19424 dermoscopic images of skin lesions captured from 2010 to 2016 in the facilities of the Hospital Clínic in Barcelona. With this dataset, we aim to study the problem of unconstrained classification of dermoscopic images of skin cancer, including lesions found in hard-to-diagnose locations (nails and mucosa), large lesions which do not fit in the aperture of the dermoscopy device, and hypo-pigmented lesions. The BCN20000 will be provided to the participants of the ISIC Challenge 2019, where they will be asked to train algorithms to c…
We propose a novel end-to-end curriculum learning approach for sparsely labelled animal datasets leveraging large volumes of unlabelled data to improve supervised species detectors. We exemplify the method in detail on the task of finding great apes in camera trap footage taken in challenging real-world jungle environments. In contrast to previous semi-supervised methods, our approach adjusts learning parameters dynamically over time and gradually improves detection quality by steering training towards virtuous self-reinforcement. To achieve this, we propose integrating pseudo-labelling with c…
Facial expression recognition (FER) in the wild is crucial for building reliable human-computer interactive systems. However, annotations of large scale datasets in FER has been a key challenge as these datasets suffer from noise due to various factors like crowd sourcing, subjectivity of annotators, poor quality of images, automatic labelling based on key word search etc. Such noisy annotations impede the performance of FER due to the memorization ability of deep networks. During early learning stage, deep networks fit on clean data. Then, eventually, they start overfitting on noisy labels du…
BuckTales : A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes
Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；University of Konstanz（康斯坦茨大学）；Indian Institute of Science（印度科学学院）；BITS Pilani Hyderabad（BITS Pilani海得拉巴校区）
SSSIHL（Sri Sathya Sai Institute of Higher Learning（斯里·萨提亚·赛高等教育学院））
Facial expression recognition(FER) in the wild is crucial for building reliable human-computer interactive systems. However, current FER systems fail to perform well under various natural and un-controlled conditions. This report presents attention based framework used in our submission to expression recognition track of the Affective Behaviour Analysis in-the-wild (ABAW) 2020 competition. Spatial-channel attention net(SCAN) is used to extract local and global attentive features without seeking any information from landmark detectors. SCAN is complemented by a complementary context information…
University of Technology Sydney（悉尼科技大学）；Baidu Inc.（百度公司）；Zhejiang University（浙江大学）
This paper explores an expression-related self-supervised learning (SSL) method (ContraWarping) to perform expression classification in the 5th Affective Behavior Analysis in-the-wild (ABAW) competition. Affective datasets are expensive to annotate, and SSL methods could learn from large-scale unlabeled data, which is more suitable for this task. By evaluating on the Aff-Wild2 dataset, we demonstrate that ContraWarping outperforms most existing supervised methods and shows great application potential in the affective analysis area. Codes will be released on: https://github.com/youqingxiaozhua/…
We propose the first metric learning system for the recognition of great ape behavioural actions. Our proposed triple stream embedding architecture works on camera trap videos taken directly in the wild and demonstrates that the utilisation of an explicit DensePose-C chimpanzee body part segmentation stream effectively complements traditional RGB appearance and optical flow streams. We evaluate system variants with different feature fusion techniques and long-tail recognition approaches. Results and ablations show performance improvements of ~12% in top-1 accuracy over previous results achieve…
Automating Visual Recognition of Leprosy in Wild Chimpanzees
University of Exeter（埃克塞特大学）；University of Bristol（布里斯托大学）；Wild Chimpanzee Foundation（野生黑猩猩基金会）；Instituto da Biodiversidade e das Áreas Protegidas (IBAP)（生物多样性和保护区研究所）
Gwangju Institute of Science and Technology（光州科学技术院）
For better photography, most recent commercial cameras including smartphones have either adopted large-aperture lens to collect more light or used a burst mode to take multiple images within short times. These interesting features lead us to examine depth from focus/defocus. In this work, we present a convolutional neural network-based depth estimation from single focal stacks. Our method differs from relevant state-of-the-art works with three unique features. First, our method allows depth maps to be inferred in an end-to-end manner even with image alignment. Second, we propose a sharp region…
Imperial College London（帝国理工学院）；University of Oulu（奥卢大学）
Automatic understanding of human affect using visual signals is a problem that has attracted significant interest over the past 20 years. However, human emotional states are quite complex. To appraise such states displayed in real-world settings, we need expressive emotional descriptors that are capable of capturing and describing this complexity. The circumplex model of affect, which is described in terms of valence (i.e., how positive or negative is an emotion) and arousal (i.e., power of the activation of the emotion), can be used for this purpose. Recent progress in the emotion recognition…
Natural human-computer interaction and audio-visual human behaviour sensing systems, which would achieve robust performance in-the-wild are more needed than ever as digital devices are increasingly becoming an indispensable part of our life. Accurately annotated real-world data are the crux in devising such systems. However, existing databases usually consider controlled settings, low demographic variability, and a single task. In this paper, we introduce the SEWA database of more than 2000 minutes of audio-visual data of 398 people coming from six cultures, 50% female, and uniformly spanning …
CHIRP dataset: towards long-term, individual-level, behavioral monitoring of bird populations in the wild
Centre for the Advanced Study of Collective Behaviour, University of Konstanz（康斯坦茨大学集体行为高级研究中心）；Dept. of Collective Behavior, Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所集体行为系）；Dept. of Biology, University of Konstanz（康斯坦茨大学生物学系）；Dept. of Computer and Information Science, University of Konstanz（康斯坦茨大学计算机与信息科学系）；Luondua Boreal Field Station（卢昂杜阿北方野外站）；School of Biological and Medical Sciences, Oxford Brookes University（牛津布鲁克斯大学生物与医学科学学院）；Dept. of Zoology, Stockholm University（斯德哥尔摩大学动物学系）；Dept. of Ecology of Animal Societies, Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所动物社会生态学系）
