Smart weeding systems to perform plant-specific operations can contribute to the sustainability of agriculture and the environment. Despite monumental advances in autonomous robotic technologies for precision weed management in recent years, work on under-canopy weeding in fields is yet to be realized. A prerequisite of such systems is reliable detection and classification of weeds to avoid mistakenly spraying and, thus, damaging the surrounding plants. Real-time multi-class weed identification enables species-specific treatment of weeds and significantly reduces the amount of herbicide use. Here, our first contribution is the first adequately large realistic image dataset \textit{AIWeeds} (one/multiple kinds of weeds in one image), a library of about 10,000 annotated images of flax, and the 14 most common weeds in fields and gardens taken from 20 different locations in North Dakota, California, and Central China. Second, we provide a full pipeline from model training with maximum efficiency to deploying the TensorRT-optimized model onto a single board computer. Based on \textit{AIWeeds} and the pipeline, we present a baseline for classification performance using five benchmark CNN models. Among them, MobileNetV2, with both the shortest inference time and lowest memory consumption, is the qualified candidate for real-time applications. Finally, we deploy MobileNetV2 onto our own compact autonomous robot \textit{SAMBot} for real-time weed detection. The 90\% test accuracy realized in previously unseen scenes in flax fields (with a row spacing of 0.2-0.3 m), with crops and weeds, distortion, blur, and shadows, is a milestone towards precision weed control in the real world. We have publicly released the dataset and code to generate the results at \url{https://github.com/StructuresComp/Multi-class-Weed-Classification}.
Selective weed treatment is a critical step in autonomous crop management as related to crop health and yield. However, a key challenge is reliable, and accurate weed detection to minimize damage to surrounding plants. In this paper, we present an approach for dense semantic weed classification with multispectral images collected by a micro aerial vehicle (MAV). We use the recently developed encoder-decoder cascaded Convolutional Neural Network (CNN), Segnet, that infers dense semantic classes while allowing any number of input image channels and class balancing with our sugar beet and weed da…
BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation
Department of Computer Science and Software Engineering, The University of Western Australia（计算机科学与软件工程系，西澳大学）；Department of Primary Industries and Regional Development (DPIRD), Government of Western Australia（初级产业与地区发展部（DPIRD），西澳政府）
High efficiency in precision farming depends on accurate tools to perform weed detection and mapping of crops. This allows for precise removal of harmful weeds with a lower amount of pesticides, as well as increase of the harvest's yield by providing the farmer with valuable information. In this paper, we address the problem of fully automatic stem detection from image data for this purpose. Our approach runs on mobile agricultural robots taking RGB images. After processing the images to obtain a vegetation mask, our approach separates each plant into its individual leaves and later estimates …
Sapienza University of Rome（罗马第一大学）；Roma Tre University（罗马第三大学）；University of Padua（帕多瓦大学）；University of Basilicata（巴西利卡塔大学）
An effective perception system is a fundamental component for farming robots, as it enables them to properly perceive the surrounding environment and to carry out targeted operations. The most recent methods make use of state-of-the-art machine learning techniques to learn a valid model for the target task. However, those techniques need a large amount of labeled data for training. A recent approach to deal with this issue is data augmentation through Generative Adversarial Networks (GANs), where entire synthetic scenes are added to the training data, thus enlarging and diversifying their info…
Applying agrochemicals is the default procedure for conventional weed control in crop production, but has negative impacts on the environment. Robots have the potential to treat every plant in the field individually and thus can reduce the required use of such chemicals. To achieve that, robots need the ability to identify crops and weeds in the field and must additionally select effective treatments. While certain types of weed can be treated mechanically, other types need to be treated by (selective) spraying. In this paper, we present an approach that provides the necessary information for …
Reducing the use of agrochemicals is an important component towards sustainable agriculture. Robots that can perform targeted weed control offer the potential to contribute to this goal, for example, through specialized weeding actions such as selective spraying or mechanical weed removal. A prerequisite of such systems is a reliable and robust plant classification system that is able to distinguish crop and weed in the field. A major challenge in this context is the fact that different fields show a large variability. Thus, classification systems have to robustly cope with substantial environ…
Agriculture is vital for human survival and remains a major driver of several economies around the world; more so in underdeveloped and developing economies. With increasing demand for food and cash crops, due to a growing global population and the challenges posed by climate change, there is a pressing need to increase farm outputs while incurring minimal costs. Previous machine vision technologies developed for selective weeding have faced the challenge of reliable and accurate weed detection. We present approaches for plant seedlings classification with a dataset that contains 4,275 images …
University of Lincoln（林肯大学）；Flanders Make（法兰德斯制造研究院）；Ghent University（根特大学）；Flanders Research Institute for Agriculture, Fisheries and Food (ILVO)（法兰德斯农业、渔业与食品研究所）；Swedish University of Agricultural Sciences（瑞典农业科学大学）
Weed and crop segmentation is becoming an increasingly integral part of precision farming that leverages the current computer vision and deep learning technologies. Research has been extensively carried out based on images captured with a camera from various platforms. Unmanned aerial vehicles (UAVs) and ground-based vehicles including agricultural robots are the two popular platforms for data collection in fields. They all contribute to site-specific weed management (SSWM) to maintain crop yield. Currently, the data from these two platforms is processed separately, though sharing the same sem…
Robotic System with AI for Real Time Weed Detection, Canopy Aware Spraying, and Droplet Pattern Evaluation
Machine Vision and Optical Sensor (MVOS) Lab, Dept. of Agricultural and Biosystems Engineering, South Dakota State University, Brookings, SD, USA（机械视觉与光学传感器实验室，农业与生物系统工程系，南达科他州立大学，布鲁金斯，SD，USA）；Department of Agronomy, Horticulture and Plant Science, South Dakota State University, Brookings, SD, USA（园艺、园艺学与植物科学系，南达科他州立大学，布鲁金斯，SD，USA）
本研究开发了一种基于AI的机器人喷洒系统，利用YOLO11模型在Jetson Orin Nano上实时检测杂草和冠层，动态调整喷嘴，实现精准除草剂施用，减少成本与污染。
Precision farming robots, which target to reduce the amount of herbicides that need to be brought out in the fields, must have the ability to identify crops and weeds in real time to trigger weeding actions. In this paper, we address the problem of CNN-based semantic segmentation of crop fields separating sugar beet plants, weeds, and background solely based on RGB data. We propose a CNN that exploits existing vegetation indexes and provides a classification in real time. Furthermore, it can be effectively re-trained to so far unseen fields with a comparably small amount of training data. We i…
Design of a six wheel suspension and a three-axis linear actuation mechanism for a laser weeding robot
Robotics and Intelligent Systems Engineering (RISE) Lab（机器人与智能系统工程实验室）；Department of Robotics and Artificial Intelligence（机器人与人工智能系）；School of Mechanical and Manufacturing Engineering (SMME)（机械与制造工程学院）；National University of Sciences and Technology (NUST)（国立科学与技术大学）
Unmanned Aerial vehicles (UAV) are a promising technology for smart farming related applications. Aerial monitoring of agriculture farms with UAV enables key decision-making pertaining to crop monitoring. Advancements in deep learning techniques have further enhanced the precision and reliability of aerial imagery based analysis. The capabilities to mount various kinds of sensors (RGB, spectral cameras) on UAV allows remote crop analysis applications such as vegetation classification and segmentation, crop counting, yield monitoring and prediction, crop mapping, weed detection, disease and nut…
AdaCropFollow: Self-Supervised Online Adaptation for Visual Under-Canopy Navigation
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Bonn（波恩大学）；University of Oxford（牛津大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
Murdoch University（莫道克大学）；Centre for Crop and Food Innovation（作物与食品创新中心）；Food Futures Institute（食品未来研究所）；Department of Primary Industries and Regional Development, Western Australia（西澳大利亚州初级产业与区域发展部）；Centre for Sustainable Farming Systems（可持续农业系统中心）；Centre of Biosecurity and One Health, Harry Butler Institute（哈里·巴特勒研究所生物安全与同一健康中心）
The rapid advances in Deep Learning (DL) techniques have enabled rapid detection, localisation, and recognition of objects from images or videos. DL techniques are now being used in many applications related to agriculture and farming. Automatic detection and classification of weeds can play an important role in weed management and so contribute to higher yields. Weed detection in crops from imagery is inherently a challenging problem because both weeds and crops have similar colours ('green-on-green'), and their shapes and texture can be very similar at the growth phase. Also, a crop in one s…
INSA CVL（INSA中央卢瓦尔河谷校区）；University of Orleans（奥尔良大学）；PRISME EA 4229（PRISME EA 4229研究所）
Crop and weed monitoring is an important challenge for agriculture and food production nowadays. Thanks to recent advances in data acquisition and computation technologies, agriculture is evolving to a more smart and precision farming to meet with the high yield and high quality crop production. Classification and recognition in Unmanned Aerial Vehicles (UAV) images are important phases for crop monitoring. Advances in deep learning models relying on Convolutional Neural Network (CNN) have achieved high performances in image classification in the agricultural domain. Despite the success of thi…
Global Rice Multi-Class Segmentation Dataset (RiceSEG): A Comprehensive and Diverse High-Resolution RGB-Annotated Images for the Development and Benchmarking of Rice Segmentation Algorithms
Nanjing Agricultural University（南京农业大学）；The University of Tokyo（东京大学）；Professor Jayashankar Telangana Agricultural State University（贾亚尚卡尔·特伦甘纳农业州立大学）；Kyoto University（京都大学）；Huazhong University of Science and Technology（华中科技大学）；Huazhong Agricultural University（华中农业大学）；Chinese Academy of Agricultural Sciences（中国农业科学院）；Chinese Academy of Science（中国科学院）；Shenyang Agricultural University（沈阳农业大学）；Jilin Academy of Agricultural Sciences（吉林省农业科学院）；Yuan Long Ping High-Tech Agriculture Co., Ltd.（袁隆平高科技农业有限公司）
AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding
China Agricultural University（中国农业大学）；Sun Yat-sen University（中山大学）；Tianjin University（天津大学）；Tsinghua University（清华大学）；Southwest Jiaotong University（西南交通大学）；National Supercomputing Center in Shenzhen（深圳国家超算中心）
