Large foundation models have revolutionized the field, yet challenges remain in optimizing multi-modal models for specialized visual tasks. We propose a novel, generalizable methodology to identify preferred image distributions for black-box Vision-Language Models (VLMs) by measuring output consistency across varied input prompts. Applying this to different rendering types of 3D objects, we demonstrate its efficacy across various domains requiring precise interpretation of complex structures, with a focus on Computer-Aided Design (CAD) as an exemplar field. We further refine VLM outputs using in-context learning with human feedback, significantly enhancing explanation quality. To address the lack of benchmarks in specialized domains, we introduce CAD-VQA, a new dataset for evaluating VLMs on CAD-related visual question answering tasks. Our evaluation of state-of-the-art VLMs on CAD-VQA establishes baseline performance levels, providing a framework for advancing VLM capabilities in complex visual reasoning tasks across various fields requiring expert-level visual interpretation. We release the dataset and evaluation codes at \url{https://github.com/asgsaeid/cad_vqa}.
Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning
University of Science and Technology Beijing（北京科技大学）；Beijing Engineering Research Center of Industrial Spectrum Imaging（北京工业光谱成像工程技术研究中心）；Sun Yat-sen University（中山大学）；Microsoft Research Asia（微软亚洲研究院）
$\left|\,\circlearrowright\,\boxed{\text{BUS}}\,\right|$: A Large and Diverse Multimodal Benchmark for evaluating the ability of Vision-Language Models to understand Rebus Puzzles
Tredence Inc.（特伦德公司）；Indian Institute of Technology Kharagpur（印度理工学院克拉格浦尔分校）；Inria Paris-Rocquencourt（巴黎-罗克琴库特研究所）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒姆研究实验室）
LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
The Hong Kong Polytechnic University（香港理工大学）；Princeton University（普林斯顿大学）；Nanjing University（南京大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving
Beihang University（北京航空航天大学）；National University of Singapore（新加坡国立大学）；Aviation Industry Development Research Center of China（中国航空工业发展研究中心）；Nanyang Technological University（南洋理工大学）
Connecting the Dots: Collaborative Fine-tuning for Black-Box Vision-Language Models
University of Science and Technology of China（中国科学技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Nanjing University（南京大学）
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding
Nanyang Technological University（南洋理工大学）；University of Reading（阅读大学）；Shanghai Jiao Tong University（上海交通大学）；Harbin Institute of Technology（哈尔滨工业大学）；SenseTime Research（商汤科技研究院）
B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructions
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家重点实验室）；National Engineering Research Center for Visual Information and Applications（视觉信息与应用国家工程研究中心）；Institute of Artificial Intelligence and Robotics（人工智能与机器人研究院）；Xi’an Jiaotong University（西安交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Osaka University（大阪大学）
Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA
School of Data Science, University of Science and Technology of China（中国科学技术大学数据科学学院）；School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；State Key Laboratory of Cognitive Intelligence（认知智能国家重点实验室）；Tencent YouTu Lab（腾讯优图实验室）
Learning to Ground Before Reading: Unified PCB Engineering Drawing Parsing with Compact Vision-Language Models
Beijing Normal–Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院）；Guangdong Provincial Key Laboratory of Interdisciplinary Research and Application for Data Science（广东省数据科学交叉研究与应用重点实验室）；Hong Kong Baptist University（香港浸会大学）；Hong Kong aiKnow Limited（香港智知有限公司）
Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Models
Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学（深圳））；Hong Kong Baptist University, China（香港 Baptist大学）；City University of Hong Kong, China（香港城市大学）
MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
A.I.MATICS Inc（A.I.MATICS公司）；Boston University（波士顿大学）；University of Southern California（南加州大学）；Heuron（Heuron公司）；MODULABS, Open Neural Networks Research Lab（MODULABS，开放神经网络研究实验室）
Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models
University of Science and Technology of China（中国科学技术大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Institute of High Performance Computing, A*STAR, Singapore（新加坡A*STAR高性能计算研究所）
