World models have emerged as promising neural simulators for autonomous driving, with the potential to supplement scarce real-world data and enable closed-loop evaluations. However, current research primarily evaluates these models based on visual realism or downstream task performance, with limited focus on fidelity to specific action instructions - a crucial property for generating targeted simulation scenes. Although some studies address action fidelity, their evaluations rely on closed-source mechanisms, limiting reproducibility. To address this gap, we develop an open-access evaluation framework, ACT-Bench, for quantifying action fidelity, along with a baseline world model, Terra. Our benchmarking framework includes a large-scale dataset pairing short context videos from nuScenes with corresponding future trajectory data, which provides conditional input for generating future video frames and enables evaluation of action fidelity for executed motions. Furthermore, Terra is trained on multiple large-scale trajectory-annotated datasets to enhance action fidelity. Leveraging this framework, we demonstrate that the state-of-the-art model does not fully adhere to given instructions, while Terra achieves improved action fidelity. All components of our benchmark framework will be made publicly available to support future research.
STAGE: STyle-controllable Action GEneration for personalized autonomous driving
Northwestern Polytechnical University(西北工业大学);School of Astronautics(航天学院);Research Center for Intelligent Robotics(智能机器人研究中心);National Key Laboratory of Aerospace Flight Dynamics(航空航天飞行动力学重点实验室)
DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents
University of California, Irvine(加州大学欧文分校);Indian Institute of Technology, Kanpur(印度理工学院坎普尔分校);Google Research(谷歌研究院)
扩散概率模型已被证明可以在几个有竞争力的图像合成基准上生成最先进的结果,但缺乏低维、可解释的潜在空间,并且生成速度很慢。另一方面,标准的变分自编码器(VAE)通常可以访问低维潜在空间,但表现出较差的样本质量。我们提出了DiffuseVAE,一个新的生成框架,集成VAE扩散模型框架内,并利用这一点来设计新的条件参数化扩散模型。我们表明,由此产生的模型装备扩散模型与低维VAE推断的潜在代码,可用于下游任务,如可控合成。所提出的方法还改进了标准无条件DDPM/DDIM模型中表现出的速度与质量权衡(例如,在CelebA-HQ-128基准上使用T=10个反向过程步骤,FID为16.47 vs 34.36),而无需针对这样的目标进行显式训练。此外,该模型的合成质量与标准图像合成基准(如CIFAR-10和CelebA-64)上的最先进模型相当,同时优于大多数现有的基于VAE的方法。最后,我们表明,所提出的方法表现出内在的推广到不同类型的噪声在调理信号。为了重现性,我们的源代码在https://github.com/kpandey008/DiffuseVAE上公开。
Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games
Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学)
PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planning
Autonomous Intelligent Systems, Computer Science Institute VI – Intelligent Systems(智能系统);Robotics, Center for Robotics(机器人学);the Lamarr Institute for Machine Learning(拉马尔机器学习研究所);Artificial Intelligence, University of Bonn, Germany(人工智能,波恩大学,德国)
Adaptive Adversarial Augmentation for Controllable Face Synthesis
Manipal University Jaipur(斋浦尔马尼帕尔大学);Birla Institute of Technology and Science Pilani Dubai Campus(Pilani贝拉理工学院迪拜校区);Indian Institute of Technology Jodhpur(焦特布尔印度理工学院)