存储更少,查找更多:新颖性过滤如何改进边缘摄像头上的跨模态检索
Storing Less, Finding More: How Novelty Filtering Improves Cross-Modal Retrieval on Edge Cameras
AI总结:
本文提出一种流式检索架构,通过边缘设备的epsilon-net过滤器保留语义新颖帧,构建去噪嵌入索引,并结合跨模态适配器和云重排序器,提升边缘摄像头跨模态检索性能。
摘要:
始终开启的边缘摄像头生成连续视频流,冗余帧会通过拥挤将正确结果挤出top-k搜索。本文提出一种流式检索架构:边缘设备的epsilon-net过滤器仅保留语义新颖帧,构建去噪嵌入索引;跨模态适配器和云重排序器补偿紧凑编码器的弱对齐。在两个眼动数据集(AEA、EPIC-KITCHENS)上,单次流式过滤器在八个视觉-语言模型(8M-632M)中优于离线方法(k-means、最远点、均匀、随机)。结合后,该架构在8M边缘编码器下达到45.6%的Hold-out数据Hit@5,功耗估计为2.7 mW。
英文摘要:
Always-on edge cameras generate continuous video streams where redundant frames degrade cross-modal retrieval by crowding correct results out of top-k search. This paper presents a streaming retrieval architecture: an on-device epsilon-net filter retains only semantically novel frames, building a denoised embedding index; a cross-modal adapter and cloud re-ranker compensate for the compact encoder's weak alignment. A single-pass streaming filter outperforms offline alternatives (k-means, farthest-point, uniform, random) across eight vision-language models (8M-632M) on two egocentric datasets (AEA, EPIC-KITCHENS). Combined, the architecture reaches 45.6% Hit@5 on held-out data using an 8M on-device encoder at an estimated 2.7 mW.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。