arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.12366cs.CV
收藏

DepthCropSeg++: 通过深度标注数据扩展作物分割基础模型

DepthCropSeg++: Scaling a Crop Segmentation Foundation Model With Depth-Labeled Data

MetaPheno Laboratory(MetaPheno实验室);PhenoTrait Technology Co., Ltd.(PhenoTrait技术有限公司);Wuhan Digital Engineering Institute(武汉数字工程研究院);National Key Laboratory of Multispectral Information Intelligent Processing Technology(国家多光谱信息智能处理技术重点实验室);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);Hubei Key Laboratory of Intelligent Robot(湖北省智能机器人重点实验室);School of Computer Science and Engineering, School of Artificial Intelligence, Wuhan Institute of Technology(武汉理工大学计算机科学与工程学院、人工智能学院)

Jiafei Zhang, Songliang Cao, Binghui Xu, Yanan Li, Weiwei Jia, Tingting Wu, Hao Lu, Weijuan Hu, Zhiguo Han

AI总结:

DepthCropSeg++通过深度标注数据扩展作物分割模型,采用两阶段自训练流程提升性能,实现93.11%的mIoU,超越监督基线和SAM模型。

摘要:

DepthCropSeg++:一种用于作物分割的基础模型,能够在开放田间环境中分割不同作物种类。作物分割是现代农业中的基本任务,与许多下游任务如植物表型分析、密度估计和杂草控制密切相关。在基础模型时代,已经开发出许多通用的大语言和视觉模型。这些模型由于显著的模型容量和大规模数据集而表现出卓越的现实世界泛化能力。然而,当前的作物分割模型大多由于像素级标注成本高昂,只能从有限的数据中学习,通常只能在特定作物类型或受控环境中表现良好。在本工作中,我们沿袭我们先前工作DepthCropSeg的思路,即一种几乎无监督的作物分割方法,以扩展跨物种和跨场景的作物分割数据集,涵盖30多种作物和15种环境条件,共计28,406张图像。我们还基于最先进的语义分割架构ViT-Adapter架构,通过动态上采样增强其细节感知能力,并通过两阶段自训练流程训练模型。为了系统验证模型性能,我们进行了全面的实验,以验证其在多个作物数据集上的有效性与泛化能力。结果表明,DepthCropSeg++在综合测试集上达到了93.11%的mIoU,显著优于监督基线和通用视觉基础模型如分割任何模型(SAM)(分别高出0.36%和48.57%)。该模型在具有挑战性的场景中表现尤为出色,包括夜间环境(86.90% mIoU)、高密度冠层(90.09% mIoU)和未见过的作物品种(90.09% mIoU),表明了作物分割的新状态-of-the-art。

英文摘要:

DepthCropSeg++: a foundation model for crop segmentation, capable of segmenting different crop species under open in-field environment. Crop segmentation is a fundamental task for modern agriculture, which closely relates to many downstream tasks such as plant phenotyping, density estimation, and weed control. In the era of foundation models, a number of generic large language and vision models have been developed. These models have demonstrated remarkable real world generalization due to significant model capacity and largescale datasets. However, current crop segmentation models mostly learn from limited data due to expensive pixel-level labelling cost, often performing well only under specific crop types or controlled environment. In this work, we follow the vein of our previous work DepthCropSeg, an almost unsupervised approach to crop segmentation, to scale up a cross-species and crossscene crop segmentation dataset, with 28,406 images across 30+ species and 15 environmental conditions. We also build upon a state-of-the-art semantic segmentation architecture ViT-Adapter architecture, enhance it with dynamic upsampling for improved detail awareness, and train the model with a two-stage selftraining pipeline. To systematically validate model performance, we conduct comprehensive experiments to justify the effectiveness and generalization capabilities across multiple crop datasets. Results demonstrate that DepthCropSeg++ achieves 93.11% mIoU on a comprehensive testing set, outperforming both supervised baselines and general-purpose vision foundation models like Segmentation Anything Model (SAM) by significant margins (+0.36% and +48.57% respectively). The model particularly excels in challenging scenarios including night-time environment (86.90% mIoU), high-density canopies (90.09% mIoU), and unseen crop varieties (90.09% mIoU), indicating a new state of the art for crop segmentation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑