Swiss DINO:面向端侧个人物品搜索的高效通用视觉框架
Swiss DINO: Efficient and Versatile Vision Framework for On-device Personal Object Search
AI总结:
针对端侧个人物品搜索的细粒度区分难、资源受限两大挑战,提出基于DINOv2的Swiss DINO框架,无需适配训练,在精度与资源效率上均显著优于现有方案。
摘要:
本文针对家用机器人设备搭载视觉系统、可实时实现设备个性化的最新趋势展开研究,具体提出并解决了个人物品搜索这一重要技术任务:该任务要求在机器人设备拍摄的图像中定位并识别用户感兴趣的个人物品,且每个物品仅需少量标注图像作为参考。该任务对需要处理个人视觉场景、或针对特定个人物品执行操作(如抓取、导航)的家用机器人与移动系统至关重要。实际应用中,个人物品搜索存在两大核心技术挑战:其一,机器人视觉系统需在存在遮挡与杂乱场景的情况下,区分大量细粒度类别;其二,端侧系统严格的资源限制,导致大多数前沿少样本学习方法难以应用,且通常无法在端侧执行适配训练。本文提出Swiss DINO:一种基于最新DINOv2 Transformer模型的简单且有效的单样本个人物品搜索框架,DINOv2已被证实具备强大的零样本泛化能力。Swiss DINO可满足端侧个性化场景理解的严苛要求,且无需任何适配训练。实验结果显示,与常见的轻量级解决方案相比,该方法在分割与识别精度上提升显著(最高达55%);与重型Transformer类解决方案相比,其骨干网络推理耗时最高降低100倍,GPU占用最高降低10倍,资源开销大幅缩减。
英文摘要:
In this paper, we address a recent trend in robotic home appliances to include vision systems on personal devices, capable of personalizing the appliances on the fly. In particular, we formulate and address an important technical task of personal object search, which involves localization and identification of personal items of interest on images captured by robotic appliances, with each item referenced only by a few annotated images. The task is crucial for robotic home appliances and mobile systems, which need to process personal visual scenes or to operate with particular personal objects (e.g., for grasping or navigation). In practice, personal object search presents two main technical challenges. First, a robot vision system needs to be able to distinguish between many fine-grained classes, in the presence of occlusions and clutter. Second, the strict resource requirements for the on-device system restrict the usage of most state-of-the-art methods for few-shot learning and often prevent on-device adaptation. In this work, we propose Swiss DINO: a simple yet effective framework for one-shot personal object search based on the recent DINOv2 transformer model, which was shown to have strong zero-shot generalization properties. Swiss DINO handles challenging on-device personalized scene understanding requirements and does not require any adaptation training. We show significant improvement (up to 55%) in segmentation and recognition accuracy compared to the common lightweight solutions, and significant footprint reduction of backbone inference time (up to 100x) and GPU consumption (up to 10x) compared to the heavy transformer-based solutions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。