UrbanAlign: Post-hoc Semantic Calibration for VLM-Human Preference Alignment
Tsinghua University(清华大学);University College London(伦敦大学学院);Hong Kong University of Science and Technology(香港科学与技术大学);Peking University(北京大学);Southwest Jiaotong University(西南交通大学);Zhejiang University(浙江大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);Renmin University of China(中国人民大学);Southeast University(东南大学)
Yecheng Zhang, Rong Zhao, Zhizhou Sha, Yong Li, Lei Wang, Ce Hou, Wen Ji, Hao Huang, Yunshan Wan, Jian Yu, Junhao Xia, Yuru Zhang, Chunlei Shi
Vision-language models (VLMs) can describe urban scenes in rich detail, yet consistently fail to produce reliable human preference labels in domain-specific tasks such as safety assessment and aesthetic evaluation. The standard fix, fine-tuning or RLHF, requires large-scale annotations and model retraining. We ask a different question: can a frozen VLM be aligned with human preferences without modifying any weights? Our key insight is that VLMs are strong concept extractors but poor decision calibrators. We propose a three-stage post-hoc pipeline that exploits this asymmetry: (i) interpretable evaluation dimensions are automatically mined from consensus exemplars; (ii) an Observer-Debater-Judge chain extracts robust concept scores from the frozen VLM; and (iii) locally-weighted ridge regression on a hybrid manifold calibrates these scores to human ratings. Applied as UrbanAlign on Place Pulse 2.0, the framework reaches 72.2% accuracy (kappa=0.45) across six perception categories, outperforming all baselines by +11.0 pp and zero-shot VLM by +15.5 pp, with full interpretability and zero weight modification.
Technical University of Munich(慕尼黑工业大学);Helmholtz Munich(亥姆霍兹慕尼黑中心);ELLIS Institute Finland(芬兰ELLIS研究所);Aalto University(阿尔托大学);Finnish Center for Artificial Intelligence(芬兰人工智能中心);Munich Center for Machine Learning(慕尼黑机器学习中心);Munich Data Science Institute(慕尼黑数据科学研究所);Ericsson Research(爱立信研究院);KTH Royal Institute of Technology(KTH皇家理工学院);Digital Futures(数字未来研究所)
VLMs Guided Interpretable Decision Making for Autonomous Driving
Department of Computer Science, Tulane University(路易斯安那大学计算机科学系);Qualcomm(高通公司);Department of Industrial and Systems Engineering, North Carolina State University(北卡罗来纳州立大学工业与系统工程系)
Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation
Johns Hopkins University, School of Medicine(约翰霍普金斯大学医学院);Massachusetts Institute of Technology(麻省理工学院);Microsoft Healthcare & Life Sciences(微软医疗健康与生命科学)
MVC-Bench: Benchmarking Calibration of Medical Vision-Language Models
Mohamed bin Zayed University of AI(穆罕默德·本·扎耶德人工智能大学);Sabaragamuwa University of Sri Lanka(斯里兰卡萨巴拉加穆瓦大学);Digital Platform Development, SLT PLC(SLT公共有限公司数字平台开发部)
Bodhi VLM: Privacy-Alignment Modeling for Hierarchical Visual Representations in Vision Backbones and VLM Encoders via Bottom-Up and Top-Down Feature Search
Auckland University of Technology(奥克兰技术大学);Resideo Technologies, Inc.(Resideo技术公司);Guilin University of Electronic Technology(桂林电子科技大学);Universidad de Chile(智利大学)
Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification
College of Computer Science, Sichuan University(四川大学计算机科学学院);Yong Loo Lin School of Medicine, National University of Singapore(新加坡国立大学杨潞龄医学院);Key Laboratory of Data Protection and Intelligent Management, Ministry of Education, Sichuan University(四川大学数据保护与智能管理教育部重点实验室);National Key Laboratory of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology(北京理工大学自主智能无人系统国家重点实验室);Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局高性能计算研究所)
Department of Computer Science, University of British Columbia(不列颠哥伦比亚大学计算机科学系);Vector Institute for AI(人工智能矢量研究所);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
Supervised Classification Heads as Semantic Prototypes: Unlocking Vision-Language Alignment via Weight Recycling
Department of Computer Science and Artificial Intelligence, DaSCI Institute, University of Granada, Granada, Spain(计算机科学与人工智能系,DaSCI研究所,格拉纳达大学,格拉纳达,西班牙);Department of Mathematics ``Tullio Levi-Civita'', University of Padova, Padova, Italy(托里利-西维塔数学系,帕多瓦大学,帕多瓦,意大利)