In this work, we present a new dataset to advance the state-of-the-art in fruit detection, segmentation, and counting in orchard environments. While there has been significant recent interest in solving these problems, the lack of a unified dataset has made it difficult to compare results. We hope to enable direct comparisons by providing a large variety of high-resolution images acquired in orchards, together with human annotations of the fruit on trees. The fruits are labeled using polygonal masks for each object instance to aid in precise object detection, localization, and segmentation. Additionally, we provide data for patch-based counting of clustered fruits. Our dataset contains over 41, 000 annotated object instances in 1000 images. We present a detailed overview of the dataset together with baseline performance analysis for bounding box detection, segmentation, and fruit counting as well as representative results for yield estimation. We make this dataset publicly available and host a CodaLab challenge to encourage comparison of results on a common dataset. To download the data and learn more about MinneApple please see the project website: http://rsn.cs.umn.edu/index.php/MinneApple. Up to date information is available online.
The localization of fruits is an essential first step in automated agricultural pipelines for yield estimation or fruit picking. One example of this is the localization of apples in images of entire apple trees. Since the apples are very small objects in such scenarios, we tackle this problem by adapting the object proposal generation system AttentionMask that focuses on small objects. We adapt AttentionMask by either adding a new module for very small apples or integrating it into a tiling framework. Both approaches clearly outperform standard object proposal generation systems on the MinneAp…
Revitalizing Dense Material Segmentation: Stabilized Vision Transformers and the Generalization Paradox
Bahcesehir University, Istanbul, Turkey（巴塞希尔大学，伊斯坦布尔，土耳其）；Poder Bilişim Teknolojileri Sanayi ve Ticaret A.Ş., Istanbul, Turkey（Poder信息科技工业和贸易股份有限公司，伊斯坦布尔，土耳其）；Galatasaray University, Istanbul, Turkey（加拉塔萨雷大学，伊斯坦布尔，土耳其）
Centrum Wiskunde & Informatica（数学与计算机科学研究所）；Leiden University（莱顿大学）；Eindhoven University of Technology（埃因霍温理工大学）
We present three parallel-beam tomographic datasets of 94 apples with internal defects along with defect label files. The datasets are prepared for development and testing of data-driven, learning-based image reconstruction, segmentation and post-processing methods. The three versions are a noiseless simulation; simulation with added Gaussian noise, and with scattering noise. The datasets are based on real 3D X-ray CT data and their subsequent volume reconstructions. The ground truth images, based on the volume reconstructions, are also available through this project. Apples contain various de…
Unified Review and Benchmark of Deep Segmentation Architectures for Cardiac Ultrasound on CAMUS
Department of Computer Science；Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea；Department of Semiconductor System Engineering, Sejong University, Seoul, 05006, Republic of Korea；Department of Operations Research；Statistics, Faculty of Organizational Sciences, University of Belgrade, Belgrade, Serbia；Department of Industrial Engineering \& Management, Yuan Ze University, Taoyuan City 320315, Taiwan；Department of Applied Mathematical Science, College of Science；Technology, Korea University, Sejong 30019, Republic of Korea
TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarios
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
Fruit tree pruning and fruit thinning require a powerful vision system that can provide high resolution segmentation of the fruit trees and their branches. However, recent works only consider the dormant season, where there are minimal occlusions on the branches or fit a polynomial curve to reconstruct branch shape and hence, losing information about branch thickness. In this work, we apply two state-of-the-art supervised learning models U-Net and DeepLabv3, and a conditional Generative Adversarial Network Pix2Pix (with and without the discriminator) to segment partially occluded 2D-open-V app…
Mendel University in Brno（布尔诺孟德尔大学）；Wageningen University & Research（瓦赫宁根大学及研究中心）
Apple scab is a fungal disease caused by Venturia inaequalis. Disease is of particular concern for growers, as it causes significant damage to fruit and leaves, leading to loss of fruit and yield. This article examines the ability of deep learning and hyperspectral imaging to accurately identify an apple symptom infection in apple trees. In total, 168 image scenes were collected using conventional RGB and Visible to Near-infrared (VIS-NIR) spectral imaging (8 channels) in infected orchards. Spectral data were preprocessed with an Artificial Neural Network (ANN) trained in segmentation to detec…
Apple is one of the remarkable fresh fruit that contains a high degree of nutritious and medicinal value. Hand harvesting of apples by seasonal farmworkers increases physical damages on the surface of these fruits, which causes a great loss in marketing quality. The main objective of this study is focused on designing a robust vision algorithm for robotic apple harvesters. The proposed algorithm is able to recognize and classify 4-classes of objects found in an orchard scene including apples, leaves, trunk and branches, and sky into two apples and non-apples classes. 100 digital images of Red …
Robotic harvesting of fruits in orchards is a challenging task, since high density and overlapping of fruits and branches can heavily impact the success rate of robotic harvesting. Therefore, the vision system is demanded to provide comprehensive information of the working environment to guide the manipulator and gripping system to successful detach the target fruits. In this study, a deep learning based one-stage detector DaSNet-V2 is developed to perform the multi-task vision sensing in the working environment of apple orchards. DaSNet-V2 combines the detection and instance segmentation of f…
Understanding Image2Video Domain Shift in Food Segmentation: An Instance-level Analysis on Apples
Interdisciplinary Program in Artificial Intelligence, Seoul National University（人工智能交叉学科项目，首尔国立大学）；Department of Biological Environmental Science, College of Life Science and Biotechnology, Dongguk University（生命科学与生物技术学院生物环境科学系，东国大学）；Department of Food Science and Biotechnology, Dongguk University（食品科学与生物技术系，东国大学）
In this research, a fully neural network based visual perception framework for autonomous apple harvesting is proposed. The proposed framework includes a multi-function neural network for fruit recognition and a Pointnet grasp estimation to determine the proper grasp pose to guide the robotic execution. Fruit recognition takes raw input of RGB images from the RGB-D camera to perform fruit detection and instance segmentation, and Pointnet grasp estimation take point cloud of each fruit as input and output the prediction of grasp pose for each of fruits. The proposed framework is validated by us…
The economic and production losses in agricultural industry worldwide are due to the presence of diseases in the several kinds of fruits. In this paper, a method for the classification of fruit diseases is proposed and experimentally validated. The image processing based proposed approach is composed of the following main steps; in the first step K-Means clustering technique is used for the defect segmentation, in the second step color and textural cues are extracted and fused from the segmented image, and finally images are classified into one of the classes by using a Multi-class Support Vec…
Diseases in fruit cause devastating problem in economic losses and production in agricultural industry worldwide. In this paper, an adaptive approach for the identification of fruit diseases is proposed and experimentally validated. The image processing based proposed approach is composed of the following main steps; in the first step K-Means clustering technique is used for the defect segmentation, in the second step some state of the art features are extracted from the segmented image, and finally images are classified into one of the classes by using a Multi-class Support Vector Machine. We…
Australian Centre for Field Robotics（澳大利亚野外机器人中心）；The University of Sydney（悉尼大学）
Ground vehicles equipped with monocular vision systems are a valuable source of high resolution image data for precision agriculture applications in orchards. This paper presents an image processing framework for fruit detection and counting using orchard image data. A general purpose image segmentation approach is used, including two feature learning algorithms; multi-scale Multi-Layered Perceptrons (MLP) and Convolutional Neural Networks (CNN). These networks were extended by including contextual information about how the image data was captured (metadata), which correlates with some of the …
Vision perception and modelling are the essential tasks of robotic harvesting in the unstructured orchard. This paper develops a framework of visual perception and modelling for robotic harvesting of fruits in the orchard environments. The developed framework includes visual perception, scenarios mapping, and fruit modelling. The Visual perception module utilises a deep-learning model to perform multi-purpose visual perception task within the working scenarios; The scenarios mapping module applies OctoMap to represent the multiple classes of objects or elements within the environment; The frui…
Jamia Millia Islamia（贾米亚·米利亚·伊斯兰亚大学）；Sher-i-Kashmir University of Agricultural Sciences（谢里·克什米尔农业科学大学）
Diseases and pests cause huge economic loss to the apple industry every year. The identification of various apple diseases is challenging for the farmers as the symptoms produced by different diseases may be very similar, and may be present simultaneously. This paper is an attempt to provide the timely and accurate detection and identification of apple diseases. In this study, we propose a deep learning based approach for identification and classification of apple diseases. The first part of the study is dataset creation which includes data collection and data labelling. Next, we train a Convo…
Marquette University（马凯特大学）；U.S. Department of Agriculture (USDA)（美国农业部）
In fruit production, critical crop management decisions are guided by bloom intensity, i.e., the number of flowers present in an orchard. Despite its importance, bloom intensity is still typically estimated by means of human visual inspection. Existing automated computer vision systems for flower identification are based on hand-engineered techniques that work only under specific conditions and with limited performance. This work proposes an automated technique for flower identification that is robust to uncontrolled environments and applicable to different flower species. Our method relies on…
Zero-Shot Automatic Annotation and Instance Segmentation using LLM-Generated Datasets: Eliminating Field Imaging and Manual Annotation for Deep Learning Model Development
South China Agricultural University（华南农业大学）；Monash University（莫纳什大学）；CSIRO（联邦科学与工业研究组织）
Semantic segmentation is a fundamental task for agricultural robots to understand the surrounding environments in natural orchards. The recent development of the LiDAR techniques enables the robot to acquire accurate range measurements of the view in the unstructured orchards. Compared to RGB images, 3D point clouds have geometrical properties. By combining the LiDAR and camera, rich information on geometries and textures can be obtained. In this work, we propose a deep-learning-based segmentation method to perform accurate semantic segmentation on fused data from a LiDAR-Camera visual sensor.…
3D Hierarchical Panoptic Segmentation in Real Orchard Environments Across Different Sensors
Center for Robotics, University of Bonn（波恩大学机器人中心）；University of Oxford（牛津大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（Lamarr机器学习与人工智能研究所）
Object detection and classification is one of the most important computer vision problems. Ever since the introduction of deep learning \cite{krizhevsky2012imagenet}, we have witnessed a dramatic increase in the accuracy of this object detection problem. However, most of these improvements have occurred using conventional 2D image processing. Recently, low-cost 3D-image sensors, such as the Microsoft Kinect (Time-of-Flight) or the Apple FaceID (Structured-Light), can provide 3D-depth or point cloud data that can be added to a convolutional neural network, acting as an extra set of dimensions. …
FruitNeRF: A Unified Neural Radiance Field based Fruit Counting Framework
Friedrich-Alexander-Universität Erlangen-Nürnberg-Fürth（弗里德里希-亚历山大-埃尔兰根-纽伦堡-菲尔特大学）；Fraunhofer Institute for Integrated Circuits (IIS) - EZRT（弗劳恩霍夫集成电路研究所（IIS）- EZRT）；University of Bamberg（班贝格大学）
