Vision-language models (VLMs) enhance the planning capability of end-to-end (E2E) driving policy by leveraging high-level semantic reasoning. However, existing approaches often overlook the dual-system consistency between VLM's high-level decision and E2E's low-level planning. As a result, the generated trajectories may misalign with the intended driving decisions, leading to weakened top-down guidance and decision-following ability of the system. To address this issue, we propose Senna-2, an advanced VLM-E2E driving policy that explicitly aligns the two systems for consistent decision-making and planning. Our method follows a consistency-oriented three-stage training paradigm. In the first stage, we conduct driving pre-training to achieve preliminary decision-making and planning, with a decision adapter transmitting VLM decisions to E2E policy in the form of implicit embeddings. In the second stage, we align the VLM and the E2E policy in an open-loop setting. In the third stage, we perform closed-loop alignment via bottom-up Hierarchical Reinforcement Learning in 3DGS environments to reinforce the safety and efficiency. Extensive experiments demonstrate that Senna-2 achieves superior dual-system consistency (19.3% F1 score improvement) and significantly enhances driving safety in both open-loop (5.7% FDE reduction) and closed-loop settings (30.6% AF-CR reduction).
GeoReason: Aligning Thinking And Answering In Remote Sensing Vision-Language Models Via Logical Consistency Reinforcement Learning
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院航空信息研究所);Key Laboratory of Target Cognition and Application Technology, Chinese Academy of Sciences(中国科学院目标认知与应用技术重点实验室);University of Chinese Academy of Sciences(中国科学院大学)
Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness
School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件学院);Nanyang Technological University, Singapore(新加坡南洋理工大学);Nanjing University(南京大学)
MonoIR-RS: Infrared Remote Sensing Vision-Language Learning with CLIP and VLM Adaptation
China University of Petroleum-Beijing at Karamay(中国石油大学(北京)克拉玛依校区);Guizhou University(贵州大学);Shenzhen Research Institute of Big Data(深圳大数据研究院);Shanghai Jiao Tong University(上海交通大学)
HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models
Zhejiang University(浙江大学);Byte Dance(字节跳动);National University of Singapore(新加坡国立大学);Angelalign Inc China(Angelalign中国公司);Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence(浙江省医学影像人工智能重点实验室)
VLM-PAR: A Vision Language Model for Pedestrian Attribute Recognition
Department of Innovation Engineering(创新工程系);University of Salento, Italy(意大利萨伦托大学);Institute of Applied Sciences and Intelligent Systems - CNR(应用科学与智能系统研究所 - CNR);University of the Basque Country UPV/EHU(巴斯克国家大学UPV/EHU);IKERBASQUE, Basque Foundation for Science(伊基塔斯克巴塞克基金会);Sorbonne University Abu Dhabi(索邦大学阿布扎比分校)
FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Driving
Shanghai Jiaotong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院);OpenDriveLab at The University of Hong Kong(香港大学OpenDrive实验室);Li Auto Inc.(Li汽车公司)
GeoWorld-VLM: Geometry from World Models for Vision-Language Models
Harvard AI and Robotics Lab(哈佛人工智能与机器人实验室);Kempner Institute for the Study of Natural and Artificial Intelligence(凯普纳自然与人工智能研究 institute);Harvard University(哈佛大学)
VLM-Aware Meta-Optic Front-End Design for Frozen Vision-Language Models
Department of Artificial Intelligence, Hanyang University(汉阳大学人工智能系);School of Computational Science and Engineering, Georgia Institute of Technology(佐治亚理工学院计算科学与工程学院);Department of Electronic Engineering, Hanyang University(汉阳大学电子工程系)
VG4D: Vision-Language Model Goes 4D Video Recognition
School of Intelligent Systems Engineering, Sun Yat-sen University(中山大学智能工程学院);Peking University(北京大学);PKU-Wuhan Institute for Artificial Intelligence(北京大学武汉人工智能研究院);ETH Zurich(苏黎世联邦理工学院)
Aligning Vision to Language: Annotation-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoning
Tongji University(同济大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);East China Normal University(华东师范大学);Stanford University(斯坦福大学);New York University(纽约大学)
本文提出 VaLiK 方法,通过跨模态信息补充构建无需注释的多模态知识图谱,提升大语言模型推理能力。
Evaluation of Safety Cognition Capability in Vision-Language Models for Autonomous Driving
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室)
DPOison-VL: Clean Preference Poisoning in Vision-Language Direct Preference Optimization
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Shenzhen Loop Area Institute(深圳河套学院);Shanghai Research Institute for Intelligent Autonomous Systems(上海智能自主系统研究院)