一种生成式AI集成的多模态框架，用于低延迟多摄像头行人重识别
A Generative AI Integrated Multimodal Framework for Low-Latency Multi-Camera Person Re-Identification
- University of Moratuwa（莫拉图瓦大学）
- Zone24x7 (Pvt) Ltd（Zone24x7（私人）有限公司）
- Chulalongkorn University（朱拉隆功大学）
- University of Colombo（科伦坡大学）
- La Trobe University（拉筹伯大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
提出一种生成式AI集成的多模态ReID框架，通过成本感知早期退出级联在模糊时触发语义和面部模态，在Market-1501和DukeMTMC-reID上以低延迟保持竞争性准确率。
AI中文摘要：
行人重识别（ReID）对于多摄像头监控与跟踪至关重要，但由于视角和光照变化、遮挡、背景杂乱以及低分辨率图像，其仍然具有挑战性。我们提出了一种生成式AI集成的多模态ReID框架，专门针对缺失线索下的鲁棒性和低延迟部署而设计。关键思想是一个成本感知的早期退出级联，它优先考虑廉价且高置信度的证据，仅在模糊情况下触发昂贵的模态。我们的系统集成了（i）来自分割行人区域的全局视觉嵌入，（ii）由视觉语言模型（VLMs）自动生成的细粒度语义属性描述，以及（iii）在面部观察可靠时可选的面部嵌入。为了优化准确性与延迟之间的平衡，我们使用成本感知的早期退出级联，而不是融合所有模态。具体来说，我们首先检查top-k检索结果，以确定查询是否无歧义。如果最佳匹配与其余候选明显分离，我们提前停止并返回结果以最小化延迟；在模糊情况下，我们保留多个假设，并调用额外的模态（面部/语义）并采用自适应可靠性加权来细化决策。我们在Market-1501和DukeMTMC-reID基准上使用mAP和Rank-1准确率报告行人重识别性能。所提出的自适应早期退出级联在不调用语义推理的情况下解决了60.7%的DukeMTMC-reID查询和68.4%的Market-1501查询，在保持竞争性检索性能的同时降低了计算开销。
英文摘要：
Person re-identification (ReID) is essential for multi-camera surveillance and tracking, yet remains difficult due to viewpoint and illumination changes, occlusion, background clutter, and low resolution imagery. We propose a generative AI integrated multimodal ReID framework designed explicitly for robustness under missing cues and low latency deployment. The key idea is a cost aware early-exit cascade that prioritizes inexpensive, high confidence evidence and only triggers expensive modalities for ambiguous cases. Our system integrates (i) global visual embeddings from segmented person regions, (ii) automatically generated fine grained semantic attribute descriptions generated by vision-language models (VLMs), and (iii) optional facial embeddings when face observations are reliable. To optimize the balance between accuracy and latency, we use a cost aware early-exit cascade instead of fusing all modalities. Specifically, we first inspect the top-k retrieval results to determine whether the query is unambiguous. If the best match is clearly separated from the remaining candidates, we stop early and return the result to minimize latency; in ambiguous cases, we keep multiple hypotheses and invoke additional modalities (face/semantic) with adaptive reliability weighting to refine the decision. We report person re-identification performance using mAP and Rank-1 accuracy on the Market-1501 and DukeMTMC-reID benchmarks. The proposed adaptive early-exit cascade resolves 60.7% of DukeMTMC-reID queries and 68.4% of Market-1501 queries without invoking semantic reasoning, reducing computational overhead while maintaining competitive retrieval performance.
