弥合姿态-语义鸿沟:基于文本的人物异常搜索的级联框架
Bridging the Pose-Semantic Gap: A Cascade Framework for Text-Based Person Anomaly Search
AI总结:
提出结构-语义解耦级联(SSDC)框架,通过两阶段检索(结构感知粗检索和多智能体语义验证)平衡效率与语义推理,在PAB基准上达到最优性能。
摘要:
基于文本的人物异常搜索利用自然语言查询从监控档案中检索特定行为事件。尽管最近的姿态感知方法能够很好地对齐几何结构,但它们面临一个根本性的姿态-语义鸿沟:语义不同的动作可能共享相似的骨骼几何结构。虽然多模态大语言模型(MLLMs)可以减少这种歧义,但将其用于大规模检索在计算上代价高昂。我们提出了结构-语义解耦级联(SSDC)框架,将检索解耦为两个阶段:(1)结构感知粗检索,其中轻量级模型通过骨骼相似性快速筛选候选;(2)侦探小组交互,一个多智能体语义验证模块。该小组包括一个用于快速二元过滤的侦探、一个用于证据提取的分析师和一个用于语义合成的写手。最后,通过将合成描述与结构先验融合,对候选进行重新排序。在PAB基准上的实验表明,SSDC通过平衡效率和语义推理实现了最先进的性能。
英文摘要:
Text-based person anomaly search retrieves specific behavioral events from surveillance archives using natural-language queries. Although recent pose-aware methods align geometric structures well, they face a fundamental Pose-Semantic Gap: semantically different actions can share similar skeletal geometries. While Multimodal Large Language Models (MLLMs) can reduce this ambiguity, using them for large-scale retrieval is computationally prohibitive. We propose the Structure-Semantic Decoupled Cascade (SSDC) framework, which decouples retrieval into two stages: (1) Structure-Aware Coarse Retrieval, where a lightweight model quickly filters candidates by skeletal similarity ; and (2) Detective Squad Interaction, a multi-agent semantic verification module. The squad consists of a Detective for fast binary filtering, an Analyst for evidence extraction, and a Writer for semantic synthesis. Finally, we re-rank candidates by fusing the synthesized captions with structural priors. Experiments on the PAB benchmark show that SSDC achieves state-of-the-art performance by balancing efficiency and semantic reasoning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。