Pollen grain micrograph classification has multiple applications in medicine and biology. Automatic pollen grain image classification can alleviate the problems of manual categorisation such as subjectivity and time constraints. While a number of computer-based methods have been introduced in the literature to perform this task, classification performance needs to be improved for these methods to be useful in practice.
In this paper, we present an ensemble approach for pollen grain microscopic image classification into four categories: Corylus Avellana well-developed pollen grain, Corylus Avellana anomalous pollen grain, Alnus well-developed pollen grain, and non-pollen (debris) instances. In our approach, we develop a classification strategy that is based on fusion of four state-of-the-art fine-tuned convolutional neural networks, namely EfficientNetB0, EfficientNetB1, EfficientNetB2 and SeResNeXt-50 deep models. These models are trained with images of three fixed sizes (224x224, 240x240, and 260x260 pixels) and their prediction probability vectors are then fused in an ensemble method to form a final classification vector for a given pollen grain image.
Our proposed method is shown to yield excellent classification performance, obtaining an accuracy of of 94.48% and a weighted F1-score of 94.54% on the ICPR 2020 Pollen Grain Classification Challenge training dataset based on five-fold cross-validation. Evaluated on the test set of the challenge, our approach achieved a very competitive performance in comparison to the top ranked approaches with an accuracy and a weighted F1-score of 96.28% and 96.30%, respectively.
In this article, we propose a general framework for multi-focal image classification and authentication, the methodology being demonstrated on microscope pollen images. The framework is meant to be generic and based on a brute force-like approach aimed to be efficient not only on any kind, and any number, of pollen images (regardless of the pollen type), but also on any kind of multi-focal images. All stages of the framework's pipeline are designed to be used in an automatic fashion. First, the optimal focus is selected using the absolute gradient method. Then, pollen grains are extracted usin…
University of Catania（卡塔尼亚大学）；University of Turin（都灵大学）
Pollen grain classification has a remarkable role in many fields from medicine to biology and agronomy. Indeed, automatic pollen grain classification is an important task for all related applications and areas. This work presents the first large-scale pollen grain image dataset, including more than 13 thousands objects. After an introduction to the problem of pollen grain classification and its motivations, the paper focuses on the employed data acquisition steps, which include aerobiological sampling, microscope image acquisition, object detection, segmentation and labelling. Furthermore, a b…
In this article, we propose an automatic method for the segmentation of pollen grains from microscope images, followed by the automatic segmentation of their exine. The objective of exine segmentation is to separate the pollen grain in two regions of interest: exine and inner part. A coarse-to-fine approach ensures a smooth and accurate segmentation of both structures. As a rough stage, grain segmentation is performed by a procedure involving clustering and morphological operations, while the exine is approximated by an iterative procedure consisting in consecutive cropping steps of the pollen…
Imperial College London（帝国理工学院）；University College London（伦敦大学学院）
We present the first unsupervised deep learning method for pollen analysis using bright-field microscopy. Using a modest dataset of 650 images of pollen grains collected from honey, we achieve family level identification of pollen. We embed images of pollen grains into a low-dimensional latent space and compare Euclidean and Riemannian metrics on these spaces for clustering. We propose this system for automated analysis of pollen and other microscopic biological structures which have only small or unlabelled datasets available.
Deep learning approaches have shown great success in image classification tasks and can aid greatly towards the fast and reliable classification of pollen grain aerial imagery. However, often-times deep learning methods in the setting of natural images can suffer generalization problems and yield poor performance on unseen test distribution. In this work, we present and a robust deep learning framework that can generalize well for pollen grain aerobiological imagery classification. We develop a convolutional neural network-based pollen grain classification approach and combine some of the best…
We propose a robust approach for performing automatic species-level recognition of fossil pollen grains in microscopy images that exploits both global shape and local texture characteristics in a patch-based matching methodology. We introduce a novel criteria for selecting meaningful and discriminative exemplar patches. We optimize this function during training using a greedy submodular function optimization framework that gives a near-optimal solution with bounded approximation error. We use these selected exemplars as a dictionary basis and propose a spatially-aware sparse coding method to m…
Geometric Data Augmentations to Mitigate Distribution Shifts in Pollen Classification from Microscopic Images
Institute of Technical Informatics, Graz University of Technology（格拉茨工业大学技术信息研究所）；University of Technology and Education, University of Da Nang（岘港大学理工大学）；Complexity Science Hub Vienna（维也纳复杂性科学中心）
Pose variation and subtle differences in appearance are key challenges to fine-grained classification. While deep networks have markedly improved general recognition, many approaches to fine-grained recognition rely on anchoring networks to parts for better accuracy. Identifying parts to find correspondence discounts pose variation so that features can be tuned to appearance. To this end previous methods have examined how to find parts and extract pose-normalized features. These methods have generally separated fine-grained recognition into stages which first localize parts using hand-engineer…
Huazhong University of Science and Technology（华中科技大学）；Alibaba Group（阿里巴巴集团）；Ant Group（蚂蚁集团）；MyBank（网商银行）
Fine-Grained Visual Classification (FGVC) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications. This paper describes our contribution at SnakeCLEF2022 with FGVC. Firstly, we design a strong multimodal backbone to utilize various meta-information to assist in fine-grained identification. Secondly, we provide new loss functions to solve the long tail distribution with dataset. Then, in order to take full advantage of unlabeled datasets, we use self-supervised learning and supervised learning joint training to…
Shenzhen University（深圳大学）；Tencent Holdings Limited（腾讯控股有限公司）；Nvidia Corporation（英伟达公司）；National Institutes of Health（美国国立卫生研究院）
Fine-grained classification of cervical cells into different abnormality levels is of great clinical importance but remains very challenging. Contrary to traditional classification methods that rely on hand-crafted or engineered features, convolution neural network (CNN) can classify cervical cells based on automatically learned deep features. However, CNN in previous studies do not involve cell morphological information, and it is unknown whether morphological features can be directly modeled by CNN to classify cervical cells. This paper presents a CNN-based method that combines cell image ap…
Department of Informatics, Systems and Communication. University of Milano - Bicocca（米兰比可卡大学信息学、系统与传播系）
In this paper we present an extensive evaluation of visual descriptors for the content-based retrieval of remote sensing (RS) images. The evaluation includes global hand-crafted, local hand-crafted, and Convolutional Neural Network (CNNs) features coupled with four different Content-Based Image Retrieval schemes. We conducted all the experiments on two publicly available datasets: the 21-class UC Merced Land Use/Land Cover (LandUse) dataset and 19-class High-resolution Satellite Scene dataset (SceneSat). The content of RS images might be quite heterogeneous, ranging from images containing fine…
Frame2Freq: Spectral Adapters for Fine-Grained Video Understanding
Institute for Artificial Intelligence, University of Stuttgart（斯图加特大学人工智能研究所）；University Hospital Heidelberg, Diagnostic and Interventional Radiology（海德堡大学医院放射诊断与介入科）；Intelligent Assistive Systems Lab, University of Hildesheim（希尔德斯海姆大学智能辅助系统实验室）
State recognition of food images can be considered as one of the promising applications of object recognition and fine-grained image classification in computer vision. In this paper, evidence is provided for the power of convolutional neural network (CNN) for food state recognition, even with a small data set. In this study, we fine-tuned a CNN initially trained on a large natural image recognition dataset (Imagenet ILSVRC) and transferred the learned feature representations to the food state recognition task. A small-scale dataset consisting of 5978 images of seven categories was constructed …
North China Electric Power University（华北电力大学）；University of Missouri（密苏里大学）
Age estimation from a single face image has been an essential task in the field of human-computer interaction and computer vision, which has a wide range of practical application values. Accuracy of age estimation of face images in the wild is relatively low for existing methods, because they only take into account the global features, while neglecting the fine-grained features of age-sensitive areas. We propose a novel method based on our attention long short-term memory (AL) network for fine-grained age estimation in the wild, inspired by the fine-grained categories and the visual attention …
Automated Palynological Analysis System: Integrating Deep Metric Learning, Detection and Classification in Bright Field Microscopy
Departamento de Física, Facultad de Ciencias Físicas y Matemáticas, Universidad de Concepción（物理系，物理与数学科学学院，康塞普西翁大学）；Facultad de Ingeniería, Universidad San Sebastián（工程学院，圣塞巴斯蒂安大学）；Facultad de Ingeniería, Universidad Andres Bello（工程学院，安德烈斯·贝洛大学）；Universidad Arturo Prat, Sede Victoria（阿尔托普大学，维多利亚校区）；Laboratory of Palynology and Plant Ecology, School of Sciences and Technologies, University of Concepción（古生物与植物生态实验室，科学与技术学院，康塞普西翁大学）
Bosch Center for Artificial Intelligence (BCAI)（博世人工智能中心）；University of Bonn（波恩大学）
To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. faulty evidence due to artefacts) by filtering gradients during optimization. The defense does not depend on human-tuned parameters. It enables explanations which are both fine-grained and preserve the c…
Horse Eye Blink Detection and Classification for Equine Affective State Assessment
Visual Analysis and Perception Lab, Aalborg University（视觉分析与感知实验室，奥尔堡大学）；Department of Animal Biosciences, Swedish University of Agricultural Sciences（动物生物科学系，瑞典农业科学大学）
New York University Tandon School of Engineering（纽约大学坦登工程学院）
Recent advances in training vision-language models have demonstrated unprecedented robustness and transfer learning effectiveness; however, standard computer vision datasets are image-only, and therefore not well adapted to such training methods. Our paper introduces a simple methodology for adapting any fine-grained image classification dataset for distributed vision-language pretraining. We implement this methodology on the challenging iNaturalist-2021 dataset, comprised of approximately 2.7 million images of macro-organisms across 10,000 classes, and achieve a new state-of-the art model in …
Prototype-Based Knowledge Guidance for Fine-Grained Structured Radiology Reporting
Computer Aided Medical Procedures, Technische Universität München, Germany（计算机辅助医疗程序，慕尼黑技术大学，德国）；Munich Center for Machine Learning, Germany（慕尼黑机器学习中心，德国）
Automatic Synthetic Data and Fine-grained Adaptive Feature Alignment for Composed Person Retrieval
Beijing University of Posts and Telecommunications（北京邮电大学）；SenseTime（商汤科技）；Beijing Key Laboratory of Network System and Network Culture（北京网络系统与网络文化重点实验室）；Key Laboratory of Intereactive Technology and Experience System, Ministry of Culture and Tourism（文化和旅游部交互技术与体验系统重点实验室）
Can VLMs Detect and Localize Fine-Grained AI-Edited Images?
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Ant Group（蚂蚁集团）；Tsinghua University（清华大学）；Shandong University（山东大学）；Wuhan University（武汉大学）
Transfer learning is a cornerstone of computer vision, yet little work has been done to evaluate the relationship between architecture and transfer. An implicit hypothesis in modern computer vision research is that models that perform better on ImageNet necessarily perform better on other vision tasks. However, this hypothesis has never been systematically tested. Here, we compare the performance of 16 classification networks on 12 image classification datasets. We find that, when networks are used as fixed feature extractors or fine-tuned, there is a strong correlation between ImageNet accura…
