arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.29631cs.CVcs.DCcs.IR

存储更少,查找更多:新颖性过滤如何改进边缘摄像头上的跨模态检索

Storing Less, Finding More: How Novelty Filtering Improves Cross-Modal Retrieval on Edge Cameras

Sherif Abdelwahab

AI总结:

本文提出一种流式检索架构,通过边缘设备的epsilon-net过滤器保留语义新颖帧,构建去噪嵌入索引,并结合跨模态适配器和云重排序器,提升边缘摄像头跨模态检索性能。

摘要:

始终开启的边缘摄像头生成连续视频流,冗余帧会通过拥挤将正确结果挤出top-k搜索。本文提出一种流式检索架构:边缘设备的epsilon-net过滤器仅保留语义新颖帧,构建去噪嵌入索引;跨模态适配器和云重排序器补偿紧凑编码器的弱对齐。在两个眼动数据集(AEA、EPIC-KITCHENS)上,单次流式过滤器在八个视觉-语言模型(8M-632M)中优于离线方法(k-means、最远点、均匀、随机)。结合后,该架构在8M边缘编码器下达到45.6%的Hold-out数据Hit@5,功耗估计为2.7 mW。

英文摘要:

Always-on edge cameras generate continuous video streams where redundant frames degrade cross-modal retrieval by crowding correct results out of top-k search. This paper presents a streaming retrieval architecture: an on-device epsilon-net filter retains only semantically novel frames, building a denoised embedding index; a cross-modal adapter and cloud re-ranker compensate for the compact encoder's weak alignment. A single-pass streaming filter outperforms offline alternatives (k-means, farthest-point, uniform, random) across eight vision-language models (8M-632M) on two egocentric datasets (AEA, EPIC-KITCHENS). Combined, the architecture reaches 45.6% Hit@5 on held-out data using an 8M on-device encoder at an estimated 2.7 mW.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑