A key challenge for much of the machine learning work on remote sensing and earth observation data is the difficulty in acquiring large amounts of accurately labeled data. This is particularly true for semantic segmentation tasks, which are much less common in the remote sensing domain because of the incredible difficulty in collecting precise, accurate, pixel-level annotations at scale. Recent efforts have addressed these challenges both through the creation of supervised datasets as well as the application of self-supervised methods. We continue these efforts on both fronts. First, we generate and release an improved version of the Agriculture-Vision dataset (Chiu et al., 2020b) to include raw, full-field imagery for greater experimental flexibility. Second, we extend this dataset with the release of 3600 large, high-resolution (10cm/pixel), full-field, red-green-blue and near-infrared images for pre-training. Third, we incorporate the Pixel-to-Propagation Module Xie et al. (2021b) originally built on the SimCLR framework into the framework of MoCo-V2 Chen et al.(2020b). Finally, we demonstrate the usefulness of this data by benchmarking different contrastive learning approaches on both downstream classification and semantic segmentation tasks. We explore both CNN and Swin Transformer Liu et al. (2021a) architectures within different frameworks based on MoCo-V2. Together, these approaches enable us to better detect key agricultural patterns of interest across a field from aerial imagery so that farmers may be alerted to problematic areas in a timely fashion to inform their management decisions. Furthermore, the release of these datasets will support numerous avenues of research for computer vision in remote sensing for agriculture.
AgroTools: A Benchmark for Tool-Augmented Multimodal Agents in Agriculture
Sun Yat-Sen University（中山大学）；Southwest University（西南大学）；Northeastern University（东北大学）；National Supercomputing Center in Shenzhen（深圳国家超算中心）；Southwest Jiaotong University（西南交通大学）；China Agricultural University（中国农业大学）；Tsinghua University（清华大学）
AgriDoctor: A Multimodal Intelligent Assistant for Agriculture
State Key Laboratory of Multimodal Artificial Intelligence Systems（多模态人工智能系统国家重点实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
University of California, Davis（加州大学戴维斯分校）；AI Institute for Next-Generation Food Systems（下一代食品系统人工智能研究所）
In recent years, deep learning models have become the standard for agricultural computer vision. Such models are typically fine-tuned to agricultural tasks using model weights that were originally fit to more general, non-agricultural datasets. This lack of agriculture-specific fine-tuning potentially increases training time and resource use, and decreases model performance, leading an overall decrease in data efficiency. To overcome this limitation, we collect a wide range of existing public datasets for three distinct tasks, standardize them, and construct standard training and evaluation pi…
Agri-LLaVA: Knowledge-Infused Large Multimodal Assistant on Agricultural Pests and Diseases
China Three Gorges University（三峡大学）；College of Computer and Information Technology, China Three Gorges University（三峡大学计算机与信息学院）；University of Birmingham（伯明翰大学）；Southern University of Science and Technology（南方科技大学）
Sensyne Health PLC（森赛恩健康公共有限公司）；Purdue University（普渡大学）
Automatic classification of pests and plants (both healthy and diseased) is of paramount importance in agriculture to improve yield. Conventional deep learning models based on convolutional neural networks require thousands of labeled examples per category. In this work we propose a method to learn from a few samples to automatically classify different pests, plants, and their diseases, using Few-Shot Learning (FSL). We learn a feature extractor to generate embeddings and then update the embeddings using Transformers. Using Mahalanobis distance, a class-covariance-based metric, we then calcula…
We introduce a labeling tool and dataset aimed to facilitate computer vision research in agriculture. The annotation tool introduces novel methods for labeling with a variety of manual, semi-automatic, and fully-automatic tools. The dataset includes original images collected from commercial greenhouses, images from PlantVillage, and images from Google Images. Images were annotated with segmentations for foreground leaf, fruit, and stem instances, and diseased leaf area. Labels were in an extended COCO format. In total the dataset contained 10k tomatoes, 7k leaves, 2k stems, and 2k diseased lea…
PAII Inc.（PAII公司）；Ping An Bank Co., Ltd（平安银行股份有限公司）
The Agriculture-Vision Challenge in CVPR is one of the most famous and competitive challenges for global researchers to break the boundary between computer vision and agriculture sectors, aiming at agricultural pattern recognition from aerial images. In this paper, we propose our solution to the third Agriculture-Vision Challenge in CVPR 2022. We leverage a data pre-processing scheme and several Transformer-based models as well as data augmentation techniques to achieve a mIoU of 0.582, accomplishing the 2nd place in this challenge.
UIUC（伊利诺伊大学厄巴纳-香槟分校）；Intelinair；University of Oregon（俄勒冈大学）
The success of deep learning in visual recognition tasks has driven advancements in multiple fields of research. Particularly, increasing attention has been drawn towards its application in agriculture. Nevertheless, while visual pattern recognition on farmlands carries enormous economic values, little progress has been made to merge computer vision and crop sciences due to the lack of suitable agricultural image datasets. Meanwhile, problems in agriculture also pose new challenges in computer vision. For example, semantic segmentation of aerial farmland images requires inference over extremel…
AgroBench: Vision-Language Model Benchmark in Agriculture
The University of Osaka（大阪大学）；Kyoto University（京都大学）；Tokyo Institute of Technology（东京工业大学）；National Institute of Advanced Industrial Science and Technology (AIST)（日本产业技术综合研究所 (AIST)）；University of Oxford（牛津大学）；OMRON SINIC X（欧姆龙思尼克斯）
Concept of a Sensor Test Environment for Dusty Agricultural Conditions
Technical University of Munich（慕尼黑工业大学）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所（MIRMI））；Baden-Wuerttemberg Cooperative State University (DHBW) Ravensburg（巴登-符腾堡州合作教育大学（DHBW）拉芬斯堡分校）
Politecnico di Torino（都灵理工大学）；LINKS Foundation（LINKS基金会）；Consorzio Interuniversitario Nazionale per l’Informatica（全国大学信息学联盟）
In this paper, we investigate the problem of Semantic Segmentation for agricultural aerial imagery. We observe that the existing methods used for this task are designed without considering two characteristics of the aerial data: (i) the top-down perspective implies that the model cannot rely on a fixed semantic structure of the scene, because the same scene may be experienced with different rotations of the sensor; (ii) there can be a strong imbalance in the distribution of semantic classes because the relevant objects of the scene may appear at extremely different scales (e.g., a field of cro…
AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Georgia Institute of Technology（佐治亚理工学院）；Johns Hopkins University（约翰斯·霍普金斯大学）
Object detection and semantic segmentation are two of the most widely adopted deep learning algorithms in agricultural applications. One of the major sources of variability in image quality acquired in the outdoors for such tasks is changing lighting condition that can alter the appearance of the objects or the contents of the entire image. While transfer learning and data augmentation to some extent reduce the need for large amount of data to train deep neural networks, the large variety of cultivars and the lack of shared datasets in agriculture makes wide-scale field deployments difficult. …
EcoWeedNet: A Lightweight and Automated Weed Detection Method for Sustainable Next-Generation Agricultural Consumer Electronics
Computer Engineering Department, King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学计算机工程系）；SDAIA-KFUPM Joint Research Center on Artificial Intelligence, IRC for Intelligent Secure Systems and Department of Computer Engineering（人工智能联合研究中心、智能安全系统研究所及计算机工程系）
This paper explores the potential for performing temporal semantic segmentation in the context of agricultural robotics without temporally labelled data. We achieve this by proposing to generate virtual temporal samples from labelled still images. By exploiting the relatively static scene and assuming that the robot (camera) moves we are able to generate virtually labelled temporal sequences with no extra annotation effort. Normally, to train a recurrent neural network (RNN), labelled samples from a video (temporal) sequence are required which is laborious and has stymied work in this directio…
Institute for Food and Agricultural Research and Technology (IRTA)（食品与农业研究技术研究所（IRTA））
Deep learning constitutes a recent, modern technique for image processing and data analysis, with promising results and large potential. As deep learning has been successfully applied in various domains, it has recently entered also the domain of agriculture. In this paper, we perform a survey of 40 research efforts that employ deep learning techniques, applied to various agricultural and food production challenges. We examine the particular agricultural problems under study, the specific models and frameworks employed, the sources, nature and pre-processing of data used, and the overall perfo…
Mississippi State University（密西西比州立大学）；Michigan State University（密歇根州立大学）；United States Department of Agriculture, Agricultural Research Service（美国农业部农业研究局）
In agricultural image analysis, optimal model performance is keenly pursued for better fulfilling visual recognition tasks (e.g., image classification, segmentation, object detection and localization), in the presence of challenges with biological variability and unstructured environments. Large-scale, balanced and ground-truthed image datasets, however, are often difficult to obtain to fuel the development of advanced, high-performance models. As artificial intelligence through deep learning is impacting analysis and modeling of agricultural images, data augmentation plays a crucial role in b…
Large Language Models and Foundation Models in Smart Agriculture: Basics, Opportunities, and Challenges
Michigan State University（密歇根州立大学）；Jeonbuk National University（国立全北大学）；North Carolina State University（北卡罗来纳州立大学）；Southeast University（东南大学）；Nanyang Technological University（南洋理工大学）
Advances in deep learning and transfer learning have paved the way for various automation classification tasks in agriculture, including plant diseases, pests, weeds, and plant species detection. However, agriculture automation still faces various challenges, such as the limited size of datasets and the absence of plant-domain-specific pretrained models. Domain specific pretrained models have shown state of art performance in various computer vision tasks including face recognition and medical imaging diagnosis. In this paper, we propose AgriNet dataset, a collection of 160k agricultural image…
UIUC（伊利诺伊大学厄巴纳-香槟分校）；Intelinair；University of Oregon（俄勒冈大学）；Agency for Defense Development（国防发展局）；DGIST（大邱庆北科学技术院）；Norwegian Computing Center（挪威计算中心）；UiT The Arctic University of Norway（特罗姆瑟大学－挪威北极大学）；Tongji University（同济大学）；Stanford University（斯坦福大学）；Chegg, Inc.（Chegg公司）；Chonnam National University（全南大学）；Chosun University（朝鲜大学）；SGGS Institute of Engineering and Technology（SGGS工程技术学院）；Tsinghua University（清华大学）
The first Agriculture-Vision Challenge aims to encourage research in developing novel and effective algorithms for agricultural pattern recognition from aerial images, especially for the semantic segmentation task associated with our challenge dataset. Around 57 participating teams from various countries compete to achieve state-of-the-art in aerial agriculture semantic segmentation. The Agriculture-Vision Challenge Dataset was employed, which comprises of 21,061 aerial and multi-spectral farmland images. This paper provides a summary of notable methods and results in the challenge. Our submis…
National University of Sciences and Technology（巴基斯坦国立科技大学）
Robots have revolutionized our way of life in recent years.One of the domains that has not yet completely benefited from the robotic automation is the agricultural sector. Agricultural Robotics should complement humans in the arduous tasks during different sub-domains of this sector. Extensive research in Agricultural Robotics has been carried out in Japan, USA, Australia and Germany focusing mainly on the heavy agricultural machinery. Pakistan is an agricultural rich country and its economy and food security are closely tied with agriculture in general and wheat in particular. However, agricu…
VLLFL: A Vision-Language Model Based Lightweight Federated Learning Framework for Smart Agriculture
Department of Electrical and Computer Engineering, The University of Alabama（电气与计算机工程系，阿拉巴马大学）；Electrical and Computer Engineering, Michigan State University（电气与计算机工程，密歇根州立大学）；Agricultural and Biological Engineering, Mississippi State University（农业与生物工程，密苏里州立大学）；Department of Computer Science, University of Alabama（计算机科学系，阿拉巴马大学）；USDA-ARS Genetics and Sustainbale Agriculture（美国农业部ARS基因与可持续农业）
National Observatory of Athens（雅典国家天文台）；National Technical University of Athens（雅典国立技术大学）
The modernization of the Common Agricultural Policy (CAP) requires the large scale and frequent monitoring of agricultural land. Towards this direction, the free and open satellite data (i.e., Sentinel missions) have been extensively used as the sources for the required high spatial and temporal resolution Earth observations. Nevertheless, monitoring the CAP at large scales constitutes a big data problem and puts a strain on CAP paying agencies that need to adapt fast in terms of infrastructure and know-how. Hence, there is a need for efficient and easy-to-use tools for the acquisition, storag…
