Building video-language foundation models is costly and difficult due to the redundant nature of video data and the lack of high-quality video-language datasets. In this paper, we propose an efficient framework to harvest video foundation models from image ones. Our method is intuitively simple by randomly dropping input video patches and masking out input text during the post-pretraining procedure. The patch dropping boosts the training efficiency significantly and text masking enforces the learning of cross-modal fusion. We conduct extensive experiments to validate the effectiveness of our method on a wide range of video-language downstream tasks including various zero-shot tasks, video question answering, and video-text retrieval. Despite its simplicity, our method achieves state-of-the-art performances, which are comparable to some heavily pretrained video foundation models. Our method is extremely efficient and can be trained in less than one day on 8 GPUs, requiring only WebVid-10M as pretraining data. We hope our method can serve as a simple yet strong counterpart for prevalent video foundation models, provide useful insights when building them, and make large pretrained models more accessible and sustainable. This is part of the InternVideo project https://github.com/OpenGVLab/InternVideo.
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
Shanghai Collaborative Innovation Center on Intelligent Visual Computing(上海智能视觉计算协同创新中心);Microsoft Cloud + AI(微软云与人工智能部门);Fudan University(复旦大学);Microsoft Research Asia(微软亚洲研究院)
Technical Report for the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Pretraining-Diverse Ensemble of Foundation Vision Encoders for Robust Outdoor Scene Understanding
LION Lab, Hefei University of Technology(合肥工业大学 LION 实验室);University of Macau(澳门大学)
Scaling Video Pretraining for Surgical Foundation Models
The Johns Hopkins University(约翰霍普金斯大学);Zhejiang University(浙江大学);Zhejiang University-University of Illinois Urbana-Champaign Institute(浙江大学伊利诺伊大学厄巴纳-香槟分校联合学院);Zhejiang Lab(之江实验室);Shaoxing University(绍兴大学);Angelalign(时代天使);Harvard Medical School(哈佛医学院)
MM-Retinal: Knowledge-Enhanced Foundational Pretraining with Fundus Image-Text Expertise
School of Computer Science and Engineering, Southeast University, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, China(教育部新一代人工智能技术及其交叉应用重点实验室);Nanjing University of Science and Technology, Nanjing, China(南京理工大学);Agency for Science, Technology and Research (A*STAR), Singapore(新加坡科技研究局)
Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining
Department of Biomedical Engineering, Southern University of Science and Technology, China(南方科技大学生物医学工程系,中国);School of Biomedical Engineering, Shenzhen University, China(深圳大学生物医学工程学院,中国)
Beyond Natural-Image Foundation Models: Benchmarking Satellite Pretraining for Ophthalmic Image Analysis
Faculty of Electrical Engineering and Computing, University of Zagreb(萨格勒布大学电气工程与计算机学院);University College London(伦敦大学学院);Institute of Ophthalmology, University College London(伦敦大学学院眼科研究所);Department of Computer Science, University College London(伦敦大学学院计算机科学系);NIHR Moorfields Biomedical Research Centre(NIHR穆尔菲尔德生物医学研究中心);Hawkes Institute, University College London(伦敦大学学院霍克斯研究所);Moorfields Eye Hospital NHS Foundation Trust(穆尔菲尔德眼科医院NHS基金会信托)
Knowledge-enhanced Pretraining for Vision-language Pathology Foundation Model on Cancer Diagnosis
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Jiao Tong University(上海交通大学);Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine(上海交通大学医学院附属新华医院);School of Artificial Intelligence(人工智能学院);Department of Pathology(病理学部);Department of Oral Pathology(口腔病理学部);Department of Pediatric Hematology/Oncology(儿科血液肿瘤科);Clinical Research and Innovation Unit(临床研究与创新单元);Department of Pediatric Cardiology(儿童心脏病科)
Lost in Distortion: Uncovering the Domain Gap Between Computer Vision and Brain Imaging -- A Study on Pretraining for Age Prediction
Center for Translational Research in Neuroimaging and Data Science(转化神经影像与数据科学研究中心);College of Artificial Intelligence, Chengdu University of Information Technology(成都信息科技大学人工智能学院);Department of Applied Mathematics and Statistics(应用数学与统计学系);McWilliams School of Biomedical Informatics(麦卡威廉生物医学信息学学院);Department of Applied Mathematics and Theoretical Physics(应用数学与理论物理系);School of Biomedical Engineering and Imaging Sciences(生物医学工程与影像科学学院)
Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);University of Washington(华盛顿大学);City University of Hong Kong (Dongguan)(香港城市大学(东莞));Medical Optical Technology R&D Center, Research Institute of Tsinghua(清华研究院医学光学技术研发中心);Jinfeng Laboratory(金凤实验室)
DIET-CP: Lightweight and Data Efficient Self Supervised Continued Pretraining
Fordham University(福特汉姆大学);Rutgers University(罗格斯大学);Pioneer Centre for AI(先锋人工智能中心);Cold Spring Harbor Laboratory(冷泉港实验室);Brown University(布朗大学);University of Copenhagen(哥本哈根大学)
TOLiD: Bridging the Architecture Gap in Vision Foundation Model to LiDAR Pretraining via Token Lifting for Distillation
SAIVT Group, School of Electrical Engineering and Robotics, Queensland University of Technology(澳大利亚昆士兰科技大学电气工程与机器人学院SAIVT组);CSIRO Robotics, CSIRO(澳大利亚联邦科学与工业研究组织机器人部)
GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data
School of Earth and Space Sciences, Peking University(地球与空间科学学院,北京大学);College of Urban and Environmental Sciences, Peking University(城市与环境科学学院,北京大学);State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation, CAS(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);Intelligent Maintenance and Operations Systems Lab(智能维护与运营系统实验室)
SonoCLIP: Mask-Guided Region-Aware Vision-Language Pretraining for Fetal Ultrasound Analysis
School of Computer Science, Wuhan University, Wuhan, China(武汉大学计算机学院,武汉,中国);Institute of Artificial Intelligence, Wuhan University, Wuhan, China(武汉大学人工智能研究院,武汉,中国);Department of Ultrasound, Renmin Hospital of Wuhan University, Wuhan, China(武汉大学仁民医院超声科,武汉,中国);National Engineering Research Center for Multimedia Software, Wuhan University, Wuhan, China(武汉大学多媒体软件国家工程研究中心,武汉,中国);Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, Wuhan, China(湖北省多媒体与网络通信工程重点实验室,武汉大学,武汉,中国)
Microsoft Health Futures UK(微软健康未来英国);Microsoft Health Futures US(微软健康未来美国);Mayo Clinic M.D(梅奥诊所);Radiology AI Lab, Mayo Clinic M.D(放射学AI实验室,梅奥诊所);Microsoft Health & Life Sciences(微软健康与生命科学);Cambridge University Hospitals M.D(剑桥大学医院)