遮挡步态识别中的专家混合：一种动作检测视角
Occluded Gait Recognition with Mixture of Experts: An Action Detection Perspective
浏览论文内容
中文总结 AI 辅助
本文提出GaitMoE，从动作检测视角利用时间与动作专家混合，应对步态识别中的遮挡问题，并构建OccGait基准，实验验证其优越性能。
中文摘要 AI 辅助
现实场景中的广泛遮挡给步态识别带来了挑战，原因在于信息缺失和噪声，以及身体在位置和尺度上的不对齐。我们认为，步态序列中丰富的动态上下文信息本身具有解决遮挡的特性：1）具有步态连续性的相邻帧允许整体身体区域推断被遮挡的身体区域；2）步态周期允许整体动作与被遮挡动作之间的信息整合。因此，我们引入一种动作检测视角，将步态序列视为动作的组成。为了在复杂遮挡场景下检测准确的动作，我们提出了一种基于动作检测的专家混合模型（GaitMoE），由时间专家混合（MTE）和动作专家混合（MAE）组成。MTE通过时间专家自适应地构建动作锚点，MAE通过动作专家从动作锚点自适应地构建动作提议。特别是，动作检测作为步态识别的代理任务，仅使用ID标签进行端到端联合训练。此外，由于缺乏统一的遮挡基准，我们构建了一个开创性的遮挡步态数据库（OccGait），包含丰富的遮挡场景和遮挡类型标注。在OccGait、OccCASIA-B、Gait3D和GREW上的大量实验证明了该方法的优越性能。代码可在https://github.com/ShiqiYu/OpenGait获取。
英文摘要
Extensive occlusions in real-world scenarios pose challenges to gait recognition due to missing and noisy information, as well as body misalignment in position and scale. We argue that rich dynamic contextual information within a gait sequence inherently possesses occlusion-solving traits: 1) Adjacent frames with gait continuity allow holistic body regions to infer occluded body regions; 2) Gait cycles allow information integration between holistic actions and occluded actions. Therefore, we introduce an action detection perspective where a gait sequence is regarded as a composition of actions. To detect accurate actions under complex occlusion scenarios, we propose an Action Detection Based Mixture of Experts (GaitMoE), consisting of Mixture of Temporal Experts (MTE) and Mixture of Action Experts (MAE). MTE adaptively constructs action anchors by temporal experts and MAE adaptively constructs action proposals from action anchors by action experts. Especially, action detection as a proxy task with gait recognition is an end-to-end joint training only with ID labels. In addition, due to the lack of a unified occluded benchmark, we construct a pioneering Occluded Gait database (OccGait), containing rich occlusion scenarios and annotations of occlusion types. Extensive experiments on OccGait, OccCASIA-B,Gait3D and GREW demonstrate the superior performance of GaitMoE.OccGait is available at https://github.com/BNU-IVC/OccGait.
发表机构
- School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）
- School of Computer Science and Technology, Beihang University（北京航空航天大学计算机科学与技术学院）
- AI Lab, Lenovo Research（联想研究院人工智能实验室）
机构由 AI 辅助整理，请以论文原文为准。
