运动学诱导的多模态三维人体姿态估计与主体级隐私保护

中文摘要 AI 辅助

多模态三维人体姿态估计（3D HPE）结合了来自RGB、LiDAR和毫米波雷达的互补信息，但基于同一批个体相关观测数据训练的模型，会引发记录级分析所忽视的隐私风险。我们提出一个统一的多模态3D HPE框架，将运动学诱导的传感器融合与主体级隐私审计和私有训练相结合。首先，我们的多模态模型对齐模态特定的关节表示，注入骨骼结构，并自适应地聚合互补的传感器证据以实现准确的姿态预测。其次，我们针对3D HPE构建了一种黑盒主体成员推断攻击，并辅以经验性的逐点最大泄漏分析，该分析刻画了单个攻击分数结果如何改变关于成员资格结果的推断。第三，我们通过动作时间分层（一种基于人群加权的受试者内采样策略，强制执行动作和时间覆盖）实例化用户级差分隐私。我们在MM-Fi数据集上跨三种不同的实验协议评估了我们的框架。源代码将在论文被接收后发布。

英文摘要

Multimodal 3D Human Pose Estimation (3D HPE) combines complementary information from RGB, LiDAR, and mmWave radar, but models trained on correlated observations from the same individuals, raise privacy risks overlooked by record level analysis. We present a unified framework for multimodal 3D HPE that couples kinematics-induced sensor fusion with subject level privacy auditing and private training. First, our multimodal model aligns modality specific joint representation, injects skeletal structure and adaptively aggregates complementary sensor evidence for accurate pose prediction. Second, we formulate a black-box subject membership inference attack for 3D HPE, complemented by an empirical pointwise maximal leakage analysis, which characterizes how individual attack score outcomes change inference about the membership outcome. Third, we instantiate user-level differential privacy via Action Temporal Stratification, a population weighted within-subject sampling strategy that enforces action and temporal coverage. We evaluate our framework on the MM-Fi dataset across three diverse experimental protocols. Source-code will be released upon acceptance.

发表机构

• University of Glasgow（格拉斯哥大学）

机构由 AI 辅助整理，请以论文原文为准。