Decades of cognitive science establish that humans navigate environments by forming cognitive maps, defined as allocentric and topology-preserving representations of 3D space. While modern Vision-Language Models (VLMs) demonstrate emergent spatial reasoning from 2D egocentric inputs, it remains unclear whether they construct an analogous 3D internal representation. In this paper, we demonstrate that current VLMs do possess a latent topological map of 3D scenes, but it is heavily overshadowed by non-geometric visual semantics, such as color and shape. By isolating this spatial subspace through cross-scene linear feature extraction, we extract a clean spatial subspace that causally controls the model's spatial outputs. We mathematically shape this latent representation and prove its correspondence to the Laplacian eigenmaps of the scene's 3D Gaussian-kernel graph, converging to the physical 3D space in the continuous limit. Motivated by this geometric identification, we further introduce a mathematically principled latent regularization method for VLMs, based on Dirichlet energy. Applying this single-term regularizer to a minimal 500-step supervised VLM fine-tuning (SFT) on simple synthetic data yields significant improvements on real-world spatial benchmarks, outperforming standard SFT and competitive baselines by up to 12.1\% in spatial tasks involving scene topology understanding. Source code is available at https://github.com/pittisl/vlm-latent-shaping
Scenes as an Attack Surface: Uncovering Scene-Grounded Safety Failures in MLLMs
Nanjing University（南京大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；China Academy of Launch Vehicle Technology（中国运载火箭技术研究院）；Beihang University（北京航空航天大学）
Recently, multiple naturalistic traffic datasets of human-driven trajectories have been published (e.g., highD, NGSim, and pNEUMA). These datasets have been used in studies that investigate variability in human driving behavior, for example for scenario-based validation of autonomous vehicle (AV) behavior, modeling driver behavior, or validating driver models. Thus far, these studies focused on the variability on an operational level (e.g., velocity profiles during a lane change), not on a tactical level (i.e., to change lanes or not). Investigating the variability on both levels is necessary …
Predicting Reaction Time to Comprehend Scenes with Foveated Scene Understanding Maps
Department of Computer Science University of California Santa Barbara（计算机科学系加州大学圣芭芭拉分校）；Graduate Program in Dynamical Neuroscience University of California Santa Barbara（动态神经科学研究生项目加州大学圣芭芭拉分校）；Department of Psychological and Brain Sciences University of California Santa Barbara（心理学与脑科学系加州大学圣芭芭拉分校）
JacobiNeRF: NeRF Shaping with Mutual Information Gradients
Tsinghua University（清华大学）；The University of Hong Kong（香港大学）；Stanford University（斯坦福大学）；NVIDIA Research（英伟达研究院）；Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Qizhi Institute（上海期智研究院）；Google Research（谷歌研究院）
Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces
Tsinghua University（清华大学）；ETH Zürich（苏黎世联邦理工学院）；MPI for Informatics（信息研究所）；Dalian University of Technology（大连理工大学）；Microsoft（微软）；Stanford University（斯坦福大学）；University of Lugano（卢加诺大学）
InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction
State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；Chohotech co. ltd.（Chohotech有限公司）；Institute of Data Science, The University of Hong Kong（香港大学数据科学研究院）；Department of Electrical and Electronic Engineering, The University of Hong Kong（香港大学电机电子工程学系）
Chinese University of Hong Kong（香港中文大学）；CUHK T Stone Robotics Institute（香港中文大学T Stone机器人研究所）；Hong Kong Centre for Logistics Robotics（香港物流机器人研究中心）；University of Oxford（牛津大学）；Tencent AI Lab（腾讯人工智能实验室）；South China University of Technology（华南理工大学）
This paper proposes a novel pretext task to address the self-supervised video representation learning problem. Specifically, given an unlabeled video clip, we compute a series of spatio-temporal statistical summaries, such as the spatial location and dominant direction of the largest motion, the spatial location and dominant color of the largest color diversity along the temporal axis, etc. Then a neural network is built and trained to yield the statistical summaries given the video frames as inputs. In order to alleviate the learning difficulty, we employ several spatial partitioning patterns…
Deep learning models obtain impressive accuracy in road scenes understanding, however they need a large quantity of labeled samples for their training. Additionally, such models do not generalise well to environments where the statistical properties of data do not perfectly match those of training scenes, and this can be a significant problem for intelligent vehicles. Hence, domain adaptation approaches have been introduced to transfer knowledge acquired on a label-abundant source domain to a related label-scarce target domain. In this work, we design and carefully analyse multiple latent spac…
Latent-Foresight: End-to-End Learning Predictable Representations for Latent World Models
Archimedes, Athena Research Center（阿基米德，雅典娜研究中心）；National Technical University of Athens（雅典国立技术大学）；University of Crete（克里特大学）；IACM-Forth（希腊研究与技术基金会计算与应用数学研究所）
Deep convolutional neural networks for semantic segmentation achieve outstanding accuracy, however they also have a couple of major drawbacks: first, they do not generalize well to distributions slightly different from the one of the training data; second, they require a huge amount of labeled data for their optimization. In this paper, we introduce feature-level space-shaping regularization strategies to reduce the domain discrepancy in semantic segmentation. In particular, for this purpose we jointly enforce a clustering objective, a perpendicularity constraint and a norm alignment goal on t…
