In recent years, vision-language models have made significant strides, excelling in tasks like optical character recognition and geometric problem-solving. However, several critical issues remain: 1) Proprietary models often lack transparency about their architectures, while open-source models need more detailed ablations of their training strategies. 2) Pre-training data in open-source works is under-explored, with datasets added empirically, making the process cumbersome. 3) Fine-tuning often focuses on adding datasets, leading to diminishing returns. To address these issues, we propose the following contributions: 1) We trained a robust baseline model using the latest advancements in vision-language models, introducing effective improvements and conducting comprehensive ablation and validation for each technique. 2) Inspired by recent work on large language models, we filtered pre-training data using perplexity, selecting the lowest perplexity data for training. This approach allowed us to train on a curated 1M dataset, achieving competitive performance. 3) During visual instruction tuning, we used model soup on different datasets when adding more datasets yielded marginal improvements. These innovations resulted in a 9B parameter model that performs competitively with state-of-the-art models. Our strategies are efficient and lightweight, making them easily adoptable by the community.
VLM Can Be a Good Assistant: Enhancing Embodied Visual Tracking with Self-Improving Vision-Language Models
State Key Laboratory of Complex & Critical Software Environment, Beihang University（北京航空航天大学复杂与关键软件环境国家重点实验室）；School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；City University of Macau（澳门城市大学）；Center on Frontiers of Computing Studies, School of Computer Science, Nat’l Eng. Research Center of Visual Technology, Peking University（北京大学计算机学院前沿计算研究中心、国家工程视觉技术研究中心）
CultureVLM: Characterizing and Improving Cultural Understanding of Vision-Language Models for over 100 Countries
University of Macau（澳门大学）；Georgia Institute of Technology（佐治亚理工学院）；Microsoft Research（微软研究院）；Squirrel AI（松鼠AI）；Lehigh University（理海大学）；William & Mary（威廉玛丽学院）
KKA: Improving Vision Anomaly Detection through Anomaly-related Knowledge from Large Language Models
School of Computer and Artificial Intelligence of Zhengzhou University（郑州大学计算机与人工智能学院）；National Supercomputing Center In Zhengzhou（郑州国家超级计算中心）；Zhejiang University（浙江大学）
Benchmarking and Improving Large Vision-Language Models for Fundamental Visual Graph Understanding and Reasoning
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China（计算与智能研究院，哈尔滨工业大学，深圳）；Peng Cheng Laboratory, Shenzhen, China（鹏城实验室，深圳）；School of Intelligence Science and Engineering, Harbin Institute of Technology, Shenzhen, China（智能科学与工程学院，哈尔滨工业大学，深圳）
Improving Large Vision-Language Models' Understanding for Flow Field Data
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）
HyperGVL: Benchmarking and Improving Large Vision-Language Models in Hypergraph Understanding and Reasoning
Southern University of Science and Technology（南方科技大学）；Hong Kong University of Science and Technology（香港科技大学）；Huawei Research（华为研究）；Beihang University（北航）
Learn to Think: Improving Multimodal Reasoning through Vision-Aware Self-Improvement Training
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence（计算机学院、多媒体软件国家工程研究中心、人工智能研究院）；Hubei Key Laboratory of Multimedia（湖北多媒体重点实验室）；Network Communication Engineering, Wuhan University, China（网络通信工程、武汉大学，中国）；The University of Sydney, Australia（悉尼大学，澳大利亚）；Nanyang Technological University, Singapore（南洋理工大学，新加坡）
GeoVLM: Improving Automated Vehicle Geolocalisation Using Vision-Language Matching
Connected and Autonomous Vehicles Lab (CAV-Lab)（连接与自动驾驶车辆实验室）；University of Surrey（萨里大学）；Centre for Vision, Speech and Signal Processing (CVSSP)（视觉、语音和信号处理中心）
TowerVision: Understanding and Improving Multilinguality in Vision-Language Models
Instituto Superior Técnico, Universidade de Lisboa（里斯本大学技术高级学院）；Instituto de Telecomunicações（电信研究所）；Carnegie Mellon University（卡内基梅隆大学）；Sword Health（Sword健康）；TransPerfect；MICS, CentraleSupélec, Université Paris-Saclay（MICS，中央圣艾尔布里大学，巴黎萨克雷大学）；ELLIS Unit Lisbon（里斯本ELLIS单位）
R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning
University of Science and Technology of China（中国科学技术大学）；NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
