An overlapping-free leaf segmentation method for plant point clouds
College of Information Sciences and Technology, Donghua University（东华大学信息科学与技术学院）；Engineering Research Center of Digitized Textile & Fashion Technology, Ministry of Education, Donghua University（东华大学数字化纺织服装技术教育部工程研究中心）
Dawei Li, Yan Cao, Guoliang Shi, Xin Cai, Yang Chen, Sifan Wang, Siyuan Yan
备注：
24 Pages, 18 Figures, 7 Tables. Intends to submit to an open-access journal
Automatic leaf segmentation, as well as identification and classification methods that built upon it, are able to provide immediate monitoring for plant growth status to guarantee the output. Although 3D plant point clouds contain abundant phenotypic features, plant leaves are usually distributed in clusters and are sometimes seriously overlapped in the canopy. Therefore, it is still a big challenge to automatically segment each individual leaf from a highly crowded plant canopy in 3D for plant phenotyping purposes. In this work, we propose an overlapping-free individual leaf segmentation method for plant point clouds using the 3D filtering and facet region growing. In order to separate leaves with different overlapping situations, we develop a new 3D joint filtering operator, which integrates a Radius-based Outlier Filter (RBOF) and a Surface Boundary Filter (SBF) to help to separate occluded leaves. By introducing the facet over-segmentation and facet-based region growing, the noise in segmentation is suppressed and labeled leaf centers can expand to their whole leaves, respectively. Our method can work on point clouds generated from three types of 3D imaging platforms, and also suitable for different kinds of plant species. In experiments, it obtains a point-level cover rate of 97% for Epipremnum aureum, 99% for Monstera deliciosa, 99% for Calathea makoyana, and 87% for Hedera nepalensis sample plants. At the leaf level, our method reaches an average Recall at 100.00%, a Precision at 99.33%, and an average F-measure at 99.66%, respectively. The proposed method can also facilitate the automatic traits estimation of each single leaf (such as the leaf area, length, and width), which has potential to become a highly effective tool for plant research and agricultural engineering.
College of Science, Beijing Forestry University（北京林业大学理学院）
The accurate classification of plant organs is a key step in monitoring the growing status and physiology of plants. A classification method was proposed to classify the leaves and stems of potted plants automatically based on the point cloud data of the plants, which is a nondestructive acquisition. The leaf point training samples were automatically extracted by using the three-dimensional convex hull algorithm, while stem point training samples were extracted by using the point density of a two-dimensional projection. The two training sets were used to classify all the points into leaf point…
College of Science, Beijing Forestry University（北京林业大学理学院）
Terrestrial laser scanning technology provides an efficient and accuracy solution for acquiring three-dimensional information of plants. The leaf-wood classification of plant point cloud data is a fundamental step for some forestry and biological research. An automatic sampling and training method for classification was proposed based on tree point cloud data. The plane fitting method was used for selecting leaf sample points and wood sample points automatically, then two local features were calculated for training and classification by using support vector machine (SVM) algorithm. The point c…
Reliable and automated 3D plant shoot segmentation is a core prerequisite for the extraction of plant phenotypic traits at the organ level. Combining deep learning and point clouds can provide effective ways to address the challenge. However, fully supervised deep learning methods require datasets to be point-wise annotated, which is extremely expensive and time-consuming. In our work, we proposed a novel weakly supervised framework, Eff-3DPSeg, for 3D plant shoot segmentation. First, high-resolution point clouds of soybean were reconstructed using a low-cost photogrammetry system, and the Mes…
College of Information and Electrical Engineering, Shenyang Agricultural University（沈阳农业大学信息与电气工程学院）；School of Information and Intelligence Engineering, University of Sanya（三亚学院信息与智能工程学院）
Nowadays, there are many approaches to acquire three-dimensional (3D) point clouds of maize plants. However, automatic stem-leaf segmentation of maize shoots from three-dimensional (3D) point clouds remains challenging, especially for new emerging leaves that are very close and wrapped together during the seedling stage. To address this issue, we propose an automatic segmentation method consisting of three main steps: skeleton extraction, coarse segmentation based on the skeleton, fine segmentation based on stem-leaf classification. The segmentation method was tested on 30 maize seedlings and …
Plant classification has a broad application prospective in agriculture and medicine, and is especially significant to the biology diversity research. As plants are vitally important for environmental protection, it is more important to identify and classify them accurately. Plant leaf classification is a technique where leaf is classified based on its different morphological features. The goal of this paper is to provide an overview of different aspects of texture based plant leaf classification and related things. At last we will be concluding about the efficient method i.e. the method that …
This paper proposes a novel framework for fluorescence plant video processing. The plant research community is interested in the leaf-level photosynthetic analysis within a plant. A prerequisite for such analysis is to segment all leaves, estimate their structures, and track them over time. We identify this as a joint multi-leaf segmentation, alignment, and tracking problem. First, leaf segmentation and alignment are applied on the last frame of a plant video to find a number of well-aligned leaf candidates. Second, leaf tracking is applied on the remaining frames with leaf candidate transform…
We present an approach to leaf level segmentation of images of Arabidopsis thaliana plants based upon detected edges. We introduce a novel approach to edge classification, which forms an important part of a method to both count the leaves and establish the leaf area of a growing plant from images obtained in a high-throughput phenotyping system. Our technique uses a relatively shallow convolutional neural network to classify image edges as background, plant edge, leaf-on-leaf edge or internal leaf noise. The edges themselves were found using the Canny edge detector and the classified edges can…
LeafTrackNet: A Deep Learning Framework for Robust Leaf Tracking in Top-Down Plant Phenotyping
Leibniz Institute for Agricultural Engineering（莱布尼茨农业工程与生物经济研究所）；BASF Belgium Coordination Center CommV（巴斯夫比利时协调中心）；BASF Metabolome Solutions GmbH（巴斯夫代谢组解决方案公司）；Technical University Berlin（柏林技术大学）；University of Potsdam（波茨坦大学）
This paper presents a novel feature set for shape-only leaf identification motivated by real-world, mobile deployment. The feature set includes basic shape features, as well as signal features extracted from local area integral invariants (LAIIs), similar to curvature maps, at multiple scales. The proposed methodology is evaluated on a number of publicly available leaf datasets with comparable results to existing methods which make use of colour and texture features in addition to shape. Over 90% classification accuracy is achieved on most datasets, with top-four accuracy for these datasets re…
We present an approach to match partially occluded plant leaves with databases of full plant leaves. Although contour based 2D shape matching has been studied extensively in the last couple of decades, matching occluded leaves with full leaf databases is an open and little worked on problem. Classifying occluded plant leaves is even more challenging than full leaf matching because of large variations and complexity of leaf structures. Matching an occluded contour with all the full contours in a database is an NP-hard problem [Su et al. ICCV2015], so our algorithm is necessarily suboptimal.
Hierarchical Approach for Joint Semantic, Plant Instance, and Leaf Instance Segmentation in the Agricultural Domain
University of Bonn（波恩大学）；University of Oxford（牛津大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）；La Sapienza University of Rome（罗马第一大学）
University of Engineering and Technology Lahore（拉合尔工程技术大学）
Identification of plant disease is usually done through visual inspection or during laboratory examination which causes delays resulting in yield loss by the time identification is complete. On the other hand, complex deep learning models perform the task with reasonable performance but due to their large size and high computational requirements, they are not suited to mobile and handheld devices. Our proposed approach contributes automated identification of plant diseases which follows a sequence of steps involving pre-processing, segmentation of diseased leaf area, calculation of features ba…
The agriculture sector is essential for every country because it provides a basic income to a large number of people and food as well, which is a fundamental requirement to survive on this planet. We see as time passes, significant changes come in the present era, which begins with Green Revolution. Due to improper knowledge of plant diseases, farmers use fertilizers in excess, which ultimately degrade the quality of food. Earlier farmers use experts to determine the type of plant disease, which was expensive and time-consuming. In today time, Image processing is used to recognize and catalog …
Robotics and Autonomous Systems The Commonwealth Scientific and Industrial Research Organisation (CSIRO), Data61 Brisbane, Australia（澳大利亚联邦科学与工业研究组织（CSIRO）数据61部布里斯班机器人与自主系统部（Robotics and Autonomous Systems））
Automated segmentation of individual leaves of a plant in an image is a prerequisite to measure more complex phenotypic traits in high-throughput phenotyping. Applying state-of-the-art machine learning approaches to tackle leaf instance segmentation requires a large amount of manually annotated training data. Currently, the benchmark datasets for leaf segmentation contain only a few hundred labeled training images. In this paper, we propose a framework for leaf instance segmentation by augmenting real plant datasets with generated synthetic images of plants inspired by domain randomisation. We…
This paper proposed a method that combines Polar Fourier Transform, color moments, and vein features to retrieve leaf images based on a leaf image. The method is very useful to help people in recognizing foliage plants. Foliage plants are plants that have various colors and unique patterns in the leaf. Therefore, the colors and its patterns are information that should be counted on in the processing of plant identification. To compare the performance of retrieving system to other result, the experiments used Flavia dataset, which is very popular in recognizing plants. The result shows that the…
University Of Edinburgh（爱丁堡大学）；IMT Lucca（卢卡IMT高等研究院）
The number of leaves a plant has is one of the key traits (phenotypes) describing its development and growth. Here, we propose an automated, deep learning based approach for counting leaves in model rosette plants. While state-of-the-art results on leaf counting with deep learning methods have recently been reported, they obtain the count as a result of leaf segmentation and thus require per-leaf (instance) segmentation to train the models (a rather strong annotation). Instead, our method treats leaf counting as a direct regression problem and thus only requires as annotation the total leaf co…
Fruit size and leaf area are important indicators for plant health and are of interest for plant nutrient management, plant protection and harvest. In this research, an image-based method for measuring the fruit volume as well as the leaf area for cabbage is presented. For this purpose, a mask region-based convolutional neural network (Mask R-CNN) was trained to segment the cabbage fruit from the leaves and assign it to the corresponding plant. The results indicated that even with a single camera, the developed method can provide a calculation accuracy of fruit size of 92.6% and an accuracy of…
We describe a new measure for the evaluation of region level segmentation of objects, as applied to evaluating the accuracy of leaf-level segmentation of plant images. The proposed approach enforces the rule that a region (e.g. a leaf) in either the image being evaluated or the ground truth image evaluated against can be mapped to no more than one region in the other image. We call this measure the subset-matched Jaccard index.
Athena Research and Innovation Center（雅典娜研究与创新中心）；Institute of Mediterranean & Forest Ecosystems（地中海与森林生态系统研究所）
Plant phenotyping refers to a quantitative description of the plants properties, however in image-based phenotyping analysis, our focus is primarily on the plants anatomical, ontogenetical and physiological properties.This technique reinforced by the success of Deep Learning in the field of image based analysis is applicable to a wide range of research areas making high-throughput screens of plants possible, reducing the time and effort needed for phenotypic characterization.In this study, we use Deep Learning methods (supervised and unsupervised learning based approaches) to semantically segm…
An Approach Towards Identifying Bangladeshi Leaf Diseases through Transfer Learning and XAI
Ahsanullah University of Science and Technology（阿沙努拉科学与技术大学）；Southeast University（东南大学）；Bangladesh University of Engineering and Technology（孟加拉工程与技术大学）
TCLeaf-Net: a transformer-convolution framework with global-local attention for robust in-field lesion-level plant leaf disease detection
School of Optoelectronic Science and Engineering, South China Normal University（光电科学与工程学院，华南师范大学）；College of Engineering Physics, Shenzhen Technology University（工程物理学院，深圳科技大学）；Faculty of Intelligent Technology, Shanghai Institute of Technology（智能技术学院，上海理工大学）；Graduate School of Information Sciences, Tohoku University（信息科学研究生院，东北大学）
Indian Institute of Information Technology Allahabad（阿拉哈巴德印度信息技术学院）
Plant leaf diseases pose a significant danger to food security and they cause depletion in quality and volume of production. Therefore accurate and timely detection of leaf disease is very important to check the loss of the crops and meet the growing food demand of the people. Conventional techniques depend on lab investigation and human skills which are generally costly and inaccessible. Recently, Deep Neural Networks have been exceptionally fruitful in image classification. In this research paper, plant leaf disease detection employing transfer learning is explored in the JPEG compressed dom…
The Alan Turing Institute（艾伦·图灵研究所）；Forschungszentrum Jülich（于利希研究中心）；University of Edinburgh（爱丁堡大学）
In recent years, there has been an increasing interest in image-based plant phenotyping, applying state-of-the-art machine learning approaches to tackle challenging problems, such as leaf segmentation (a multi-instance problem) and counting. Most of these algorithms need labelled data to learn a model for the task at hand. Despite the recent release of a few plant phenotyping datasets, large annotated plant image datasets for the purpose of training deep learning algorithms are lacking. One common approach to alleviate the lack of training data is dataset augmentation. Herein, we propose an al…
China Agricultural University（中国农业大学）；University of California, Berkeley（加州大学伯克利分校）；LuDong University（鲁东大学）
Leaf segmentation is the most direct and effective way for high-throughput plant phenotype data analysis and quantitative researches of complex traits. Currently, the primary goal of plant phenotyping is to raise the accuracy of the autonomous phenotypic measurement. In this work, we present the LeafMask neural network, a new end-to-end model to delineate each leaf region and count the number of leaves, with two main components: 1) the mask assembly module merging position-sensitive bases of each predicted box after non-maximum suppression (NMS) and corresponding coefficients to generate origi…
Deep learning techniques involving image processing and data analysis are constantly evolving. Many domains adapt these techniques for object segmentation, instantiation and classification. Recently, agricultural industries adopted those techniques in order to bring automation to farmers around the globe. One analysis procedure required for automatic visual inspection in this domain is leaf count and segmentation. Collecting labeled data from field crops and greenhouses is a complicated task due to the large variety of crops, growth seasons, climate changes, phenotype diversity, and more, espe…
Plants are fundamentally important to life. Key research areas in plant science include plant species identification, weed classification using hyper spectral images, monitoring plant health and tracing leaf growth, and the semantic interpretation of leaf information. Botanists easily identify plant species by discriminating between the shape of the leaf, tip, base, leaf margin and leaf vein, as well as the texture of the leaf and the arrangement of leaflets of compound leaves. Because of the increasing demand for experts and calls for biodiversity, there is a need for intelligent systems that…
University of Saskatchewan（萨斯喀彻温大学）；Queen’s University（女王大学）
Manual determination of plant phenotypic properties such as plant architecture, growth, and health is very time consuming and sometimes destructive. Automatic image analysis has become a popular approach. This research aims to identify the position (and number) of leaves from a temporal sequence of high-quality indoor images consisting of multiple views, focussing in particular of images of maize. The procedure used a segmentation on the images, using the convex hull to pick the best view at each time step, followed by a skeletonization of the corresponding image. To remove skeleton spurs, a d…
Stockholm University（斯德哥尔摩大学）；Amirkabir University of Technology (Tehran Polytechnic)（阿米尔卡比尔理工大学（德黑兰理工大学））
Modern scientific and technological advances allow botanists to use computer vision-based approaches for plant identification tasks. These approaches have their own challenges. Leaf classification is a computer-vision task performed for the automated identification of plant species, a serious challenge due to variations in leaf morphology, including its size, texture, shape, and venation. Researchers have recently become more inclined toward deep learning-based methods rather than conventional feature-based methods due to the popularity and successful implementation of deep learning methods in…
Pontifical Catholic University of Paraná（巴拉那州天主教大学）；State University of Ponta Grossa（蓬塔格罗萨州立大学）；Federal University of Paraná（巴拉那联邦大学）
Automatic plant classification is a challenging problem due to the wide biodiversity of the existing plant species in a fine-grained scenario. Powerful deep learning architectures have been used to improve the classification performance in such a fine-grained problem, but usually building models that are highly dependent on a large training dataset and which are not scalable. In this paper, we propose a novel method based on a two-view leaf image representation and a hierarchical classification strategy for fine-grained recognition of plant species. It uses the botanical taxonomy as a basis fo…
In this paper, we investigate the problem of counting rosette leaves from an RGB image, an important task in plant phenotyping. We propose a data-driven approach for this task generalized over different plant species and imaging setups. To accomplish this task, we use state-of-the-art deep learning architectures: a deconvolutional network for initial segmentation and a convolutional network for leaf counting. Evaluation is performed on the leaf counting challenge dataset at CVPPP-2017. Despite the small number of training samples in this dataset, as compared to typical deep learning image sets…
Plant species identification is time consuming, costly, and requires lots of efforts, and expertise knowledge. In recent, many researchers use deep learning methods to classify plants directly using plant images. While deep learning models have achieved a great success, the lack of interpretability limit their widespread application. To overcome this, we explore the use of interpretable, measurable and computer-aided features extracted from plant leaf images. Image processing is one of the most challenging, and crucial steps in feature-extraction. The purpose of image processing is to improve …
Deakin University（迪肯大学）；Lancaster University（兰卡斯特大学）；The University of Warwick（华威大学）；South China University of Technology（华南理工大学）；Medigrowth Australia Pty Ltd（澳大利亚Medigrowth私人有限公司）
As an essential prerequisite task in image-based plant phenotyping, leaf segmentation has garnered increasing attention in recent years. While self-supervised learning is emerging as an effective alternative to various computer vision tasks, its adaptation for image-based plant phenotyping remains rather unexplored. In this work, we present a self-supervised leaf segmentation framework consisting of a self-supervised semantic segmentation model, a color-based leaf segmentation algorithm, and a self-supervised color correction model. The self-supervised semantic segmentation model groups the se…
ViTaL: An Advanced Framework for Automated Plant Disease Identification in Leaf Images Using Vision Transformers and Linear Projection For Feature Reduction
Plant phenotyping tasks such as leaf segmentation and counting are fundamental to the study of phenotypic traits. Since it is well-suited for these tasks, deep supervised learning has been prevalent in recent works proposing better performing models at segmenting and counting leaves. Despite good efforts from research groups, one of the main challenges for proposing better methods is still the limitation of labelled data availability. The main efforts of the field seem to be augmenting existing limited data sets, and some aspects of the modelling process have been under-discussed. This paper e…
Leaf vein forms the basis of leaf characterization and classification. Different species have different leaf vein patterns. It is seen that leaf vein segmentation will help in maintaining a record of all the leaves according to their specific pattern of veins thus provide an effective way to retrieve and store information regarding various plant species in database as well as provide an effective means to characterize plants on the basis of leaf vein structure which is unique for every species. The algorithm proposes a new way of segmentation of leaf veins with the use of Odd Gabor filters and…
Several researches in leaf identification did not include color information as features. The main reason is caused by a fact that they used green colored leaves as samples. However, for foliage plants, plants with colorful leaves, fancy patterns in their leaves, and interesting plants with unique shape, color and also texture could not be neglected. For example, Epipremnum pinnatum 'Aureum' and Epipremnum pinnatum 'Marble Queen' have similar patterns, same shape, but different colors. Combination of shape, color, texture features, and other attribute contained on the leaf is very useful in lea…
