基于基础模型与图神经网络的少样本结构感知机械零件分割
Few-shot Structure-Informed Machinery Part Segmentation with Foundation Models and Graph Neural Networks
AI总结：
本研究提出结合CLIPSeg、SAM等基础模型与GCN的少样本机械零件分割方法，仅需少量标注样本即可实现精准分割，训练快且泛化性强，为自主系统相关应用提供了有效方案。
摘要：
本文针对具有空间与层级关系的多零件机械结构，提出了一种新颖的少样本语义分割方法。该方法将基础模型CLIPSeg、Segment Anything Model（SAM，分割一切模型）与兴趣点检测器SuperPoint、图卷积网络（GCN）相结合，以实现机械零件的精准分割。研究在纯合成的车载装载起重机数据集上进行评估，仅需提供1至25个标注样本，模型即可在不同细节层级下实现有效分割，且在消费级GPU上的训练时间控制在5分钟以内。该模型展现出对真实数据的强泛化能力：使用10个合成支持样本时，在真实数据上的合成到真实泛化定性评估中取得92.2的J&F分数；在DAVIS 2017数据集上进行基准测试时，仅用3个支持样本便在半监督视频分割任务中达到71.5的J&F分数。该方法训练速度快且对真实数据泛化效果好，是与机械和基础设施交互的自主系统的实用工具，也彰显了基础模型的组合协同在少样本分割任务中的应用潜力。
英文摘要：
This paper proposes a novel approach to few-shot semantic segmentation for machinery with multiple parts that exhibit spatial and hierarchical relationships. Our method integrates the foundation models CLIPSeg and Segment Anything Model (SAM) with the interest point detector SuperPoint and a graph convolutional network (GCN) to accurately segment machinery parts. By providing 1 to 25 annotated samples, our model, evaluated on a purely synthetic dataset depicting a truck-mounted loading crane, achieves effective segmentation across various levels of detail. Training times are kept under five minutes on consumer GPUs. The model demonstrates robust generalization to real data, achieving a qualitative synthetic-to-real generalization with a $J\&F$ score of 92.2 on real data using 10 synthetic support samples. When benchmarked on the DAVIS 2017 dataset, it achieves a $J\&F$ score of 71.5 in semi-supervised video segmentation with three support samples. This method's fast training times and effective generalization to real data make it a valuable tool for autonomous systems interacting with machinery and infrastructure, and illustrate the potential of combined and orchestrated foundation models for few-shot segmentation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
