In the vast and dynamic landscape of urban settings, Traffic Safety Description and Analysis plays a pivotal role in applications ranging from insurance inspection to accident prevention. This paper introduces CityLLaVA, a novel fine-tuning framework for Visual Language Models (VLMs) designed for urban scenarios. CityLLaVA enhances model comprehension and prediction accuracy through (1) employing bounding boxes for optimal visual data preprocessing, including video best-view selection and visual prompt engineering during both training and testing phases; (2) constructing concise Question-Answer sequences and designing textual prompts to refine instruction comprehension; (3) implementing block expansion to fine-tune large VLMs efficiently; and (4) advancing prediction accuracy via a unique sequential questioning-based prediction augmentation. Demonstrating top-tier performance, our method achieved a benchmark score of 33.4308, securing the leading position on the leaderboard. The code can be found: https://github.com/alibaba/AICITY2024_Track2_AliOpenTrek_CityLLaVA
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments
College of Systems Engineering, National University of Defense Technology(系统工程学院,国防科技大学);State Key Laboratory of Digital Intelligent Modeling and Simulation(数字智能建模与仿真国家重点实验室);BNRist, Tsinghua University(清华大学北京研究院);Department of Electronic Engineering, Tsinghua University(电子工程系,清华大学)
Unleashing the Capabilities of Large Vision-Language Models for Intelligent Perception of Roadside Infrastructure
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS)(信息工程测绘遥感国家重点实验室);Wuhan University(武汉大学);Hubei Luojia Laboratory(湖北珞珈实验室)
ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation
Tianmushan Laboratory, Beihang University(北航之梦实验室,北京航空航天大学);Hangzhou International Innovation Institute, Beihang University(杭州国际创新研究院,北京航空航天大学);Foshan Graduate School of Innovation, Northeastern University(佛山创新研究生学院,东北大学);School of Aeronautic Science and Engineering, Beihang University(航空科学与工程学院,北京航空航天大学);Faculty of Robot Science and Engineering, Northeastern University(机器人科学与工程学院,东北大学)
STER-VLM: Spatio-Temporal With Enhanced Reference Vision-Language Models
Ho Chi Minh University of Technology(胡志明理工大学);Vietnamese-German University(越德大学);Ho Chi Minh University of Science(胡志明理工大学);University of Information Technology(信息科技大学);Vietnam National University, Ho Chi Minh city(越南国家大学,胡志明市)