Estimating human and camera trajectories with accurate scale in the world coordinate system from a monocular video is a highly desirable yet challenging and ill-posed problem. In this study, we aim to recover expressive parametric human models (i.e., SMPL-X) and corresponding camera poses jointly, by leveraging the synergy between three critical players: the world, the human, and the camera. Our approach is founded on two key observations. Firstly, camera-frame SMPL-X estimation methods readily recover absolute human depth. Secondly, human motions inherently provide absolute spatial cues. By integrating these insights, we introduce a novel framework, referred to as WHAC, to facilitate world-grounded expressive human pose and shape estimation (EHPS) alongside camera pose estimation, without relying on traditional optimization techniques. Additionally, we present a new synthetic dataset, WHAC-A-Mole, which includes accurately annotated humans and cameras, and features diverse interactive human motions as well as realistic camera trajectories. Extensive experiments on both standard and newly established benchmarks highlight the superiority and efficacy of our framework. We will make the code and dataset publicly available.
Beyond Gaussian Bottlenecks: Topologically Aligned Encoding of Vision-Transformer Feature Spaces
Department of Computer Engineering, Koç University, Istanbul, Turkey(科克大学计算机工程系,伊斯坦布尔,土耳其);Department of Computer Engineering, Hacettepe University, Ankara, Turkey(哈恰塔佩大学计算机工程系,安卡拉,土耳其);KUIS AI Research Center, Istanbul, Turkey(KUIS人工智能研究中心,伊斯坦布尔,土耳其);Department of Electrical and Electronics Engineering, Koç University, Istanbul, Turkey(科克大学电气与电子工程系,伊斯坦布尔,土耳其)
OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models
USTC(中国科学技术大学);CASIA(中国科学院自动化研究所);HKUST(香港科技大学);Infrec;Tsinghua University(清华大学);UCAS(中国科学院大学);The Chinese University of Hong Kong(香港中文大学);Sun Yat-sen University(中山大学);University of Waterloo(滑铁卢大学);Jiangnan University(江南大学);NUS(新加坡国立大学);Fiveages;SUTD(新加坡科技设计大学)
GaussVid: Sparse-View Gaussian Splatting with 3D-Aware Video Diffusion Priors
The University of Hong Kong(香港大学);School of Computing and Data Science, The University of Hong Kong(香港大学计算与数据科学学院);Futurewei Technologies Inc(华为主流技术公司)
Seeing through Light and Darkness: Sensor-Physics Grounded Deblurring HDR NeRF from Single-Exposure Images and Events
State Key Laboratory of Virtual Reality Technology and Systems, SCSE & QRI, Beihang University(北京航空航天大学虚拟现实技术与系统国家重点实验室);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
SGDet3D++: Geometry-Grounded Semantics for 4D Radar and Camera 3D Object Detection
College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息科学与电子工程学院);School of Automotive Studies, Tongji University(同济大学汽车学院)
Estimating the Diameter at Breast Height of Trees in a Forest from RGB
General Robotics, Automation, Sensing and Perception (GRASP) Laboratory, University of Pennsylvania(通用机器人、自动化、传感与感知实验室,宾夕法尼亚大学);Department of Forest Resources and Environmental Conservation, Virginia Tech(森林资源与环境保护系,弗吉尼亚理工学院)
PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perception
Harvard AI and Robotics Lab, Harvard University(哈佛人工智能与机器人实验室,哈佛大学);Media Lab and Electrical Engineering and Computer Science, Massachusetts Institute of Technology(媒体实验室和电气工程与计算机科学,麻省理工学院);Department of Computing, Imperial College London(计算系,帝国理工学院);Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University(哈佛大学自然与人工智能研究学院)
Video Models as Native 4D Renderers: World-Grounded Conditioning from Animated Mesh
Tsinghua University(清华大学);The Hong Kong Polytechnic University(香港理工大学);University of Electronic Science and Technology of China(电子科技大学);Sun Yat-sen University(中山大学);The University of Hong Kong(香港大学);University of Science and Technology of China(中国科学技术大学);Nanyang Technological University(南洋理工大学)
Field Converter: Geometry-Initialized Temporal Residual Refinement for World-Grounded Player Pose Estimation from Soccer Broadcasts
Institut de Biomécanique Humaine Georges Charpak (IBHGC), Arts et Métiers ParisTech(乔治·沙帕克人体生物力学研究所(IBHGC),巴黎高科国立高等工艺学校);French Football Federation (FFF)(法国足球联合会(FFF))