Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment
Northwestern Polytechnical University(西北工业大学);Southern University of Science and Technology(南方科技大学);The University of Sheffield(谢菲尔德大学);Hengqin Laboratory(横琴实验室);Tsinghua University(清华大学)
Jingkun Chen, Ruoshi Xu, Mingqi Gao, Shengda Luo, Jungong Han
Point-Vision-Language Models promise to empower embodied agents with executable spatial reasoning, yet they frequently succumb to geometric hallucination where predicted 3D structures contradict the observed 2D reality. We identify a key cause of this failure not as a representation bottleneck but as a structural misalignment in reinforcement learning, where sparse geometric tokens are drowned out by noisy and broadcasted sequence-level rewards. To resolve this causal dilution, we propose Geometric Reward Credit Assignment, a framework that disentangles holistic supervision into field-specific signals and routes them exclusively to their responsible token spans. This mechanism transforms vague feedback into precise gradient updates and effectively turns generic policy optimization into targeted structural alignment. Furthermore, we internalize physical constraints via a Reprojection-Consistency term which serves as a cross-modal verifier to penalize physically impossible geometries. Validated on a calibrated benchmark derived from ShapeNetCore, our approach bridges the reliability gap by boosting 3D KPA from 0.64 to 0.93, increasing 3D bounding box intersection over union to 0.686, and raising reprojection consistency scores to 0.852. Crucially, these gains are achieved while maintaining robust 2D localization performance, marking a meaningful step from plausible textual outputs toward physically verifiable spatial predictions.
Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning
Joy Future Academy(京东探索研究院);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);The University of Hong Kong(香港大学)
Strategic Fusion of Vision Language Models: Shapley-Credited Context-Aware Dawid-Skene for Multi-Label Tasks in Autonomous Driving
Centre for Transport Engineering and Modelling, Department of Civil and Environmental Engineering, Imperial College London(交通工程与建模中心,土木与环境工程系,帝国理工学院伦敦分校);ELM Europe(ELM欧洲)
Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales
University of Central Florida(中央佛罗里达大学);University of California, Santa Cruz(加州大学圣克鲁兹分校);Columbia University(哥伦比亚大学);Amazon Research(亚马逊研究);University of California, Berkeley(加州大学伯克利分校)