While current methods have shown promising progress on estimating 3D human motion from monocular videos, their motion estimates are often physically unrealistic because they mainly consider kinematics. In this paper, we introduce Physics-aware Pretrained Transformer (PhysPT), which improves kinematics-based motion estimates and infers motion forces. PhysPT exploits a Transformer encoder-decoder backbone to effectively learn human dynamics in a self-supervised manner. Moreover, it incorporates physics principles governing human motion. Specifically, we build a physics-based body representation and contact force model. We leverage them to impose novel physics-inspired training losses (i.e., force loss, contact loss, and Euler-Lagrange loss), enabling PhysPT to capture physical properties of the human body and the forces it experiences. Experiments demonstrate that, once trained, PhysPT can be directly applied to kinematics-based estimates to significantly enhance their physical plausibility and generate favourable motion forces. Furthermore, we show that these physically meaningful quantities translate into improved accuracy of an important downstream task: human action recognition.
LISMARS；Hubei Luojia Laboratory（湖北珞珈实验室）；Wuhan University（武汉大学）；The University of Hong Kong（香港大学）；The Hong Kong Polytechnic University（香港理工大学）；Texas A&M University（德克萨斯农工大学）
MV-FCOS3D++: Multi-View Camera-Only 4D Object Detection with Pretrained Monocular Backbones
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Nanyang Technological University（南洋理工大学）；The Chinese University of Hong Kong（香港中文大学）；Hong Kong University of Science and Technology（香港科技大学）
在这份技术报告中，我们介绍了我们的解决方案，称为MV-FCOS 3D ++，用于Waymo开放数据集挑战赛2022中的仅摄像头3D检测轨道。对于多视图仅相机3D检测，基于鸟瞰视图或3D几何表示的方法可以利用来自相邻视图之间的重叠区域的立体提示，并且直接执行3D检测而无需手工制作的后处理。然而，它缺乏对2D主干的直接语义监督，这可以通过预训练简单的基于单眼的检测器来补充。我们的解决方案是一个多视图框架，4D检测遵循这种模式。它基于一个简单的单目检测器FCOS 3D ++，仅使用Waymo的对象注释进行预训练，并将多视图特征转换为3D网格空间以检测其上的3D对象。设计了一种用于单帧理解和时间立体匹配的双路径颈部，以结合多帧信息。我们的方法最终使用单个模型实现了49.75%的mAPL，并在WOD挑战中获得第二名，在训练期间没有任何基于LiDAR的深度监督。该代码将在https://github.com/Tai-Wang/Depth-from-Motion上发布。
No Pose Estimation? No Problem: Pose-Agnostic and Instance-Aware Test-Time Adaptation for Monocular Depth Estimation
Department of Artificial Intelligence, Kyungpook National University（人工智能系，庆北国立大学）；Department of Electrical and Computer Engineering, Queen’s University（电气与计算机工程系，皇后大学）；Department of Information and Communications Engineering, Pukyong National University（信息与通信工程系，浦项国立大学）
FlowHMR: Physically Plausible Motion Capture from Video
Peking University（北京大学）；The Hong Kong University of Science and Technology（香港科技大学）；Tencent（腾讯）；East China Normal University（华东师范大学）；Sun Yat-sen University（中山大学）；Zhejiang University（浙江大学）
GraphiContact: Pose-aware Human-Scene Robust Contact Perception for Interactive Systems
Tsinghua University, China（清华大学, 中国）；XR System Application Research Center, Nanchang Research Institute, Zhejiang University, China（浙江大学南昌研究院XR系统应用研究中心, 中国）；Beijing University of Posts and Telecommunications, China（北京邮电大学, 中国）；University of Chinese Academy of Sciences, China（中国科学院大学, 中国）；National University of Singapore, Singapore（新加坡国立大学, 新加坡）
