Supervised video pretraining is a common transfer learning practice for improving downstream action recognition performance. However, it requires large-scale labeled source datasets, and the effectiveness of the learned initialization is influenced by the similarity between the source and target domains. Constructing such labeled pretraining datasets for different target domains is costly and difficult to scale. To address these limitations, this study proposes a label-efficient video learning framework that combines annotation-free video pretraining with target-label-set-aware fine-tuning. During pretraining, a vision-language model (VLM) generates textual descriptions of unlabeled videos, which are processed to construct an interpretable semantic pseudo-label space. A frozen video-language model then produces zero-shot soft target distributions over this space, allowing a student video encoder to learn semantically rich representations without manual source annotations. During downstream adaptation, target-label-set-aware fine-tuning combines supervised learning from labeled target videos with zero-shot distillation over the actual target label set, helping preserve VLM-derived semantic guidance while adapting the pretrained encoder to the target task. Experiments on UCF101 and HMDB51 show that the proposed framework outperforms the compared semi-supervised video action recognition methods across all evaluated limited-label regimes. Moreover, the annotation-free pretraining stage learns transferable representations that provide an effective initialization for full-data fine-tuning, despite relying on a comparatively modest unlabeled pretraining pool.
DAIT: Distillation from Vision-Language Models to Lightweight Classifiers with Adaptive Intermediate Teacher Transfer
School of Computer and Electronic Information, Nanjing Normal University, Nanjing, China（南京师范大学计算机与电子信息学院）；Medical Artificial Intelligence Lab, Westlake University（西湖大学医学人工智能实验室）
PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；International Digital Economy Academy (IDEA)（国际数字经济学院（IDEA））；The Hong Kong Polytechnic University（香港理工大学）；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）
Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs
Department of Information Technology（信息科技系）；Electrical Engineering, ETH Zurich, Zurich, Switzerland（电气工程，苏黎世联邦理工学院，苏黎世，瑞士）；Qualcomm AI Research, Amsterdam, the Netherlands（高通人工智能研究，阿姆斯特丹，荷兰）；Department of Electrical, Electronic and Information Engineering（电气、电子与信息工程系）；University of Bologna, Bologna, Italy（博洛尼亚大学，博洛尼亚，意大利）；School of Electrical and Electronic Engineering（电气与电子工程学院）
VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation
Tuebingen AI Center（图宾根人工智能中心）；University of Tuebingen（图宾根大学）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；Inria, École Normale Supérieure, CNRS, PSL Research University（法国国家科学研究院、巴黎-萨克勒大学、École Normale Supérieure、PSL研究大学）
Bridging Weakly-Supervised Learning and VLM Distillation: Noisy Partial Label Learning for Efficient Downstream Adaptation
Institute of Perceptual Intelligence, Peng Cheng Laboratory（感知智能研究院，鹏城实验室）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院，清华大学）
VLM-CPL: Consensus Pseudo Labels from Vision-Language Models for Annotation-Free Pathological Image Classification
School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China（电子科技大学机械与电子工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Department of Pathology, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Affiliated Cancer Hospital of University of Electronic Science and Technology of China（pathology department, 四川省癌症临床研究中心, 四川省肿瘤医院及研究所, 电子科技大学附属肿瘤医院）；Department of Radiation Oncology, Sichuan Cancer Hospital and Institute, University of Electronic Science and Technology of China（放射肿瘤科, 四川省肿瘤医院及研究所, 电子科技大学）
Beihang University（北京航空航天大学）；ETH Zurich（苏黎世联邦理工学院）；Bytedance AI Lab（字节跳动人工智能实验室）
Pre-trained vision-language models (VLMs) have achieved impressive results in a range of vision-language tasks. However, popular VLMs usually consist of hundreds of millions of parameters which brings challenges for fine-tuning and deployment in real-world applications due to space, memory, and latency constraints. In this work, we introduce a distilling then pruning framework to compress large vision-language models into smaller, faster, and more accurate ones. We first shrink the size of a pre-trained large VLM and apply knowledge distillation in the vision-language pre-training stage to obt…
DocVAL: Validated Chain-of-Thought Distillation for Grounded Document VQA
Department of Computer Science（计算机科学系）；Engineering, Ohio State University, Ohio, US（工程系，俄亥俄州立大学，俄亥俄，美国）；Department of Computer Science, University of Central Florida, Florida, US（计算机科学系，中央佛罗里达大学，佛罗里达，美国）
Growing a Multi-head Twig via Distillation and Reinforcement Learning to Accelerate Large Vision-Language Models
Zhejiang Key Laboratory of Space Information Sensing and Transmission, School of Computer Science, Hangzhou Dianzi University（浙江省空间信息感知与传输重点实验室，计算机学院，杭州电子科技大学）；Li Auto Inc.（理想汽车）；School of Intelligence Science and Engineering, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）智能科学与工程学院）
Adversarial Prompt Distillation for Vision-Language Models
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University（上海智能信息处理实验室，计算机科学学院，复旦大学）；The Chinese University of Hong Kong, Shatin, Hong Kong（香港中文大学，沙田，香港）；The University of Hong Kong, Pokfulam, Hong Kong（香港大学，薄扶林，香港）
Vision-Language Model Purified Semi-Supervised Semantic Segmentation for Remote Sensing Images
Faculty of Data Science, City University of Macau（数据科学学院，澳门城市大学）；School of Automation, Southeast University（自动化学院，东南大学）；College of Oceanography and Space Informatics, China University of Petroleum (East China)（海洋与空间信息学院，中国石油大学（华东））
BioGait-VLM: A Tri-Modal Vision-Language-Biomechanics Framework for Interpretable Clinical Gait Assessment
Department of Computer Science, Drexel University（德克萨斯大学计算机科学系）；Department of Neurological Surgery, Washington University（华盛顿大学神经外科系）；University of California, Berkeley（加州大学伯克利分校）
GeoWorld-VLM: Geometry from World Models for Vision-Language Models
Harvard AI and Robotics Lab（哈佛人工智能与机器人实验室）；Kempner Institute for the Study of Natural and Artificial Intelligence（凯普纳自然与人工智能研究 institute）；Harvard University（哈佛大学）
Weak to Strong: VLM-Based Pseudo-Labeling as a Weakly Supervised Training Strategy in Multimodal Video-based Hidden Emotion Understanding Tasks
The University of New South Wales, Sydney, New South Wales, Australia（新南威尔士大学）；The University of Sydney, Sydney, New South Wales, Australia（悉尼大学）；School of Software and Microelectronics, Peking University, Zibo, Shandong, China（北京大学软件与微电子学院）；Shandong University of Technology, Beijing, China（山东理工大学）
Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University, China（通用人工智能国家重点实验室，智能科学与技术学院，北京大学，中国）；Huawei Noah’s Ark Lab, China（华为诺亚方舟实验室，中国）
