Automatic plant classification is a challenging problem due to the wide biodiversity of the existing plant species in a fine-grained scenario. Powerful deep learning architectures have been used to improve the classification performance in such a fine-grained problem, but usually building models that are highly dependent on a large training dataset and which are not scalable. In this paper, we propose a novel method based on a two-view leaf image representation and a hierarchical classification strategy for fine-grained recognition of plant species. It uses the botanical taxonomy as a basis for a coarse-to-fine strategy applied to identify the plant genus and species. The two-view representation provides complementary global and local features of leaf images. A deep metric based on Siamese convolutional neural networks is used to reduce the dependence on a large number of training samples and make the method scalable to new plant species. The experimental results on two challenging fine-grained datasets of leaf images (i.e. LifeCLEF 2015 and LeafSnap) have shown the effectiveness of the proposed method, which achieved recognition accuracy of 0.87 and 0.96 respectively.
Advances in deep learning and transfer learning have paved the way for various automation classification tasks in agriculture, including plant diseases, pests, weeds, and plant species detection. However, agriculture automation still faces various challenges, such as the limited size of datasets and the absence of plant-domain-specific pretrained models. Domain specific pretrained models have shown state of art performance in various computer vision tasks including face recognition and medical imaging diagnosis. In this paper, we propose AgriNet dataset, a collection of 160k agricultural image…
Self-Supervised Learning of Plant Image Representations
INRIA；LIRMM；Université de Montpellier（蒙彼利埃大学）；Institut National de l’Audiovisuel（法国国家视听研究所）；AMAP；CIRAD（国际热带农业研究中心）；INRAE（法国国家农业食品与环境研究中心）；IRD（法国国际研究与发展中心）
Plant species identification is time consuming, costly, and requires lots of efforts, and expertise knowledge. In recent, many researchers use deep learning methods to classify plants directly using plant images. While deep learning models have achieved a great success, the lack of interpretability limit their widespread application. To overcome this, we explore the use of interpretable, measurable and computer-aided features extracted from plant leaf images. Image processing is one of the most challenging, and crucial steps in feature-extraction. The purpose of image processing is to improve …
The Commonwealth Scientific and Industrial Research Organisation (CSIRO), Data61（联邦科学与工业研究组织（CSIRO）Data61）
The traditional paradigm of applying deep learning -- collect, annotate and train on data -- is not applicable to image-based plant phenotyping as almost 400,000 different plant species exists. Data costs include growing physical samples, imaging and labelling them. Model performance is impacted by the species gap between the domain of each plant species, it is not generalisable and may not transfer to unseen plant species. In this paper, we investigate the use of synthetic data for leaf instance segmentation. We study multiple synthetic data training regimes using Mask-RCNN when few or no ann…
ETH Zürich（苏黎世联邦理工学院）；WSL（瑞士联邦森林、雪和景观研究所）；University of Zurich（苏黎世大学）
Automatic identification of plant specimens from amateur photographs could improve species range maps, thus supporting ecosystems research as well as conservation efforts. However, classifying plant specimens based on image data alone is challenging: some species exhibit large variations in visual appearance, while at the same time different species are often visually similar; additionally, species observations follow a highly imbalanced, long-tailed distribution due to differences in abundance as well as observer biases. On the other hand, most species observations are accompanied by side inf…
Plant species identification in the wild is a difficult problem in part due to the high variability of the input data, but also because of complications induced by the long-tail effects of the datasets distribution. Inspired by the most recent fine-grained visual classification approaches which are based on attention to mitigate the effects of data variability, we explore the idea of using object detection as a form of attention. We introduce a bottom-up approach based on detecting plant organs and fusing the predictions of a variable number of organ-based species classifiers. We also curate a…
PlantSegNeRF: A few-shot, cross-species method for plant 3D instance point cloud reconstruction via joint-channel NeRF with multi-view image instance matching
This paper investigates the issue of real-world identification to fulfill better species protection. We focus on plant species identification as it is a classic and hot issue. In tradition plant species identification the samples are scanned specimen and the background is simple. However, real-world species recognition is more challenging. We first systematically investigate what is realistic species recognition and the difference from tradition plant species recognition. To deal with the challenging task, an interdisciplinary collaboration is presented based on the latest advances in computer…
Recently, the EAGL-I system was developed to rapidly create massive labeled datasets of plants intended to be commonly used by farmers and researchers to create AI-driven solutions in agriculture. As a result, a publicly available plant species recognition dataset composed of 40,000 images with different sizes consisting of 8 plant species was created with the system in order to demonstrate its capabilities. This paper proposes a novel method, called Variably Overlapping Time-Coherent Sliding Window (VOTCSW), that transforms a dataset composed of images with variable size to a 3D representatio…
Indian Institute of Information Technology, Bhopal（博帕尔印度信息技术学院）；Maulana Azad National Institute of Technology, Bhopal（博帕尔毛拉纳·阿扎德国立理工学院）
Deep learning is currently the most important branch of machine learning, with applications in speech recognition, computer vision, image classification, and medical imaging analysis. Plant recognition is one of the areas where image classification can be used to identify plant species through their leaves. Botanists devote a significant amount of time to recognizing plant species by personally inspecting. This paper describes a method for dissecting color images of Swedish leaves and identifying plant species. To achieve higher accuracy, the task is completed using transfer learning with the …
The University of Danang - University of Science and Technology（岘港大学-科技大学）；The University of Danang - VN-UK Institute for Research and Executive Education（岘港大学-越南英国研究与教育学院）
It is complicated to distinguish among thousands of plant species in the natural ecosystem, and many efforts have been investigated to address the issue. In Vietnam, the task of identifying one from 12,000 species requires specialized experts in flora management, with thorough training skills and in-depth knowledge. Therefore, with the advance of machine learning, automatic plant identification systems have been proposed to benefit various stakeholders, including botanists, pharmaceutical laboratories, taxonomists, forestry services, and organizations. The concept has fueled an interest in res…
Supervised learning is often used to count objects in images, but for counting small, densely located objects, the required image annotations are burdensome to collect. Counting plant organs for image-based plant phenotyping falls within this category. Object counting in plant images is further challenged by having plant image datasets with significant domain shift due to different experimental conditions, e.g. applying an annotated dataset of indoor plant images for use on outdoor images, or on a different plant species. In this paper, we propose a domain-adversarial learning approach for dom…
Google Research（谷歌研究院）；Google Inc.（谷歌公司）；Google Earth Outreach（谷歌地球推广）；Conservation International（国际自然保护协会）
Wildlife monitoring is crucial to nature conservation and has been done by manual observations from motion-triggered camera traps deployed in the field. Widespread adoption of such in-situ sensors has resulted in unprecedented data volumes being collected over the last decade. A significant challenge exists to process and reliably identify what is in these images efficiently. Advances in computer vision are poised to provide effective solutions with custom AI models built to automatically identify images of interest and label the species in them. Here we outline the data unification effort for…
TasselNetV4: A vision foundation model for cross-scene, cross-scale, and cross-species plant counting
National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation（multispectral information intelligent processing technology, school of artificial intelligence and automation）；China-Poland Joint Laboratory on Measurement and Control Technology, School of Artificial Intelligence and Automation；School of Cyber Science and Engineering；School of Computer Science and Engineering；Wuhan Institute of Technology；Graduate School of Agricultural and Life Sciences；Engineering Research Center of Plant Phenotyping, Ministry of Education, Jiangsu Collaborative Innovation Center for Modern Crop Production, Academy for Advanced Interdisciplinary Studies
Plants are fundamentally important to life. Key research areas in plant science include plant species identification, weed classification using hyper spectral images, monitoring plant health and tracing leaf growth, and the semantic interpretation of leaf information. Botanists easily identify plant species by discriminating between the shape of the leaf, tip, base, leaf margin and leaf vein, as well as the texture of the leaf and the arrangement of leaflets of compound leaves. Because of the increasing demand for experts and calls for biodiversity, there is a need for intelligent systems that…
University of Malaya（马来亚大学）；Royal Botanic Gardens, Kew（英国皇家植物园邱园）；Kingston University（金斯顿大学）
This paper studies convolutional neural networks (CNN) to learn unsupervised feature representations for 44 different plant species, collected at the Royal Botanic Gardens, Kew, England. To gain intuition on the chosen features from the CNN model (opposed to a 'black box' solution), a visualisation technique based on the deconvolutional networks (DN) is utilized. It is found that venations of different order have been chosen to uniquely represent each of the plant species. Experimental results using these CNN features with different classifiers show consistency and superiority compared to the …
Purdue University（普渡大学）；University of Tübingen（蒂宾根大学）；Indiana University - Purdue University（印第安纳大学-普渡大学）
Fine-grained zero-shot learning task requires some form of side-information to transfer discriminative information from seen to unseen classes. As manually annotated visual attributes are extremely costly and often impractical to obtain for a large number of classes, in this study we use DNA as side information for the first time for fine-grained zero-shot classification of species. Mitochondrial DNA plays an important role as a genetic marker in evolutionary biology and has been used to achieve near-perfect accuracy in the species classification of living organisms. We implement a simple hier…
Friedrich Schiller University Jena（耶拿弗里德里希·席勒大学）；Max Planck Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）
Monitoring the responses of plants to environmental changes is essential for plant biodiversity research. This, however, is currently still being done manually by botanists in the field. This work is very laborious, and the data obtained is, though following a standardized method to estimate plant coverage, usually subjective and has a coarse temporal resolution. To remedy these caveats, we investigate approaches using convolutional neural networks (CNNs) to automatically extract the relevant data from images, focusing on plant community composition and species coverages of 9 herbaceous plant …
Zero-Shot 3D Plant Organ Segmentation with SAM3 and Semantic NeRFs
Fraunhofer Institute for Integrated Circuits (IIS)（弗劳恩霍夫集成电路研究所）；Otto-Friedrich-Universität Bamberg（班贝格奥托·弗里德里希大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
Tomato Multi-Angle Multi-Pose Dataset for Fine-Grained Phenotyping
Institute for Breeding Research on Horticultural Crops, Julius Kuehn-Institute（园艺作物育种研究所，朱利叶斯·克恩研究所）；Computer Graphics Group, Center for Sensor Systems (ZESS), University of Siegen（计算机图形组，传感器系统中心（ZESS），希尔德斯海姆大学）
Data-driven Prediction of Species-Specific Plant Responses to Spectral-Shifting Films from Leaf Phenotypic and Photosynthetic Traits
Smart Farm Research Center, Korea Institute of Science and Technology（韩国科学技术院智能农场研究中心）；Research Institute of Agriculture and Life Sciences, Seoul National University（首尔国立大学农业与生命科学研究院）；Department of Agriculture, Forestry and Bioresources, Seoul National University（首尔国立大学农业、林业与生物资源系）
School of Engineering, Griffith University（格里菲斯大学工程学院）；School of Computer Science and Technology, Wuhan University of Technology（武汉理工大学计算机科学与技术学院）；Northeast Institute of Geography and Agroecology, Chinese Academy of Sciences（中国科学院东北地理与农业生态研究所）
Leaf image recognition techniques have been actively researched for plant species identification. However it remains unclear whether leaf patterns can provide sufficient information for cultivar recognition. This paper reports the first attempt on soybean cultivar recognition from plant leaves which is not only a challenging research problem but also important for soybean cultivar evaluation, selection and production in agriculture. In this paper, we propose a novel multiscale sliding chord matching (MSCM) approach to extract leaf patterns that are distinctive for soybean cultivar identificati…
King AbdulAziz University（阿卜杜勒阿齐兹大学）；IISRI, Deakin University（迪肯大学IISRI）；Smartify Pty Ltd（Smartify私人有限公司）；Harvard University（哈佛大学）
This study aims to explore different pre-trained models offered in the Torchvision package which is available in the PyTorch library. And investigate their effectiveness on fine-grained images classification. Transfer Learning is an effective method of achieving extremely good performance with insufficient training data. In many real-world situations, people cannot collect sufficient data required to train a deep neural network model efficiently. Transfer Learning models are pre-trained on a large data set, and can bring a good performance on smaller datasets with significantly lower training …
The Institute for Sustainable Industries and Liveable Cities (ISILC), Victoria University（维多利亚大学可持续产业与宜居城市研究所（ISILC））；Namal Institute（纳马尔学院）；Charles Sturt University（查尔斯特大学）；Victoria University（维多利亚大学）
In the Agriculture sector, control of plant leaf diseases is crucial as it influences the quality and production of plant species with an impact on the economy of any country. Therefore, automated identification and classification of plant leaf disease at an early stage is essential to reduce economic loss and to conserve the specific species. Previously, to detect and classify plant leaf disease, various Machine Learning models have been proposed; however, they lack usability due to hardware incompatibility, limited scalability and inefficiency in practical usage. Our proposed DeepLens Classi…
Prior work on plant species classification predominantly focuses on building models from isolated plant attributes. Hence, there is a need for tools that can assist in species identification in the natural world. We present a novel and robust two-fold approach capable of identifying trees in a real-world natural setting. Further, we leverage unlabelled data through deep semi-supervised learning and demonstrate superior performance to supervised learning. Our single-GPU implementation for feature recognition uses minimal annotated data and achieves accuracies of 93.96% and 93.11% for leaves and…
Understanding Representation Gaps Across Scales in Tropical Tree Species Classification from Drone Imagery
Mila – Quebec AI Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）；Université de Montréal（蒙特利尔大学）；Cary Institute of Ecosystem Studies（生态系统研究所）；Smithsonian Tropical Research Institute（史密松国家热带研究所）；Department of Plant Sciences, University of Cambridge（剑桥大学植物科学系）；Universidade do Estado do Mato Grosso (UNEMAT)（马托格罗索州立大学（UNEMAT））
Instituto de Fisica de Cantabria (CSIC-UC)（坎塔布里亚物理研究所 (CSIC-UC)）
This paper discusses the potential of applying deep learning techniques for plant classification and its usage for citizen science in large-scale biodiversity monitoring. We show that plant classification using near state-of-the-art convolutional network architectures like ResNet50 achieves significant improvements in accuracy compared to the most widespread plant classification application in test sets composed of thousands of different species labels. We find that the predictions can be confidently used as a baseline classification in citizen science communities like iNaturalist (or its Span…
Kansas State University（堪萨斯州立大学）；The Land Institute（土地研究所）
A common requirement of plant breeding programs across the country is companion planting -- growing different species of plants in close proximity so they can mutually benefit each other. However, the determination of companion plants requires meticulous monitoring of plant growth. The technique of ocular monitoring is often laborious and error prone. The availability of image processing techniques can be used to address the challenge of plant growth monitoring and provide robust solutions that assist plant scientists to identify companion plants. This paper presents a new image processing alg…
Weed scouting is an important part of modern integrated weed management but can be time consuming and sparse when performed manually. Automated weed scouting and weed destruction has typically been performed using classification systems able to classify a set group of species known a priori. This greatly limits deployability as classification systems must be retrained for any field with a different set of weed species present within them. In order to overcome this limitation, this paper works towards developing a clustering approach to weed scouting which can be utilized in any field without t…
We present two large datasets of labelled plant-images that are suited towards the training of machine learning and computer vision models. The first dataset encompasses as the day of writing over 1.2 million images of indoor-grown crops and weeds common to the Canadian Prairies and many US states. The second dataset consists of over 540,000 images of plants imaged in farmland. All indoor plant images are labelled by species and we provide rich etadata on the level of individual images. This comprehensive database allows to filter the datasets under user-defined specifications such as for exam…
University of Southern Denmark, The Maersk Mc-Kinney Moller Institute（南丹麦大学，马士基麦克-金尼穆勒研究所）；Aarhus University, Department of Engineering - Signal Processing（奥胡斯大学，工程学院 - 信号处理）；Aarhus University, Department of Agroecology - Crop Health（奥胡斯大学，农业生态学系 - 作物健康）
A database of images of approximately 960 unique plants belonging to 12 species at several growth stages is made publicly available. It comprises annotated RGB images with a physical resolution of roughly 10 pixels per mm. To standardise the evaluation of classification results obtained with the database, a benchmark based on $f_{1}$ scores is proposed. The dataset is available at https://vision.eng.au.dk/plant-seedlings-dataset
