Low-resource settings are well-established in natural language processing, where many languages lack sufficient data for deep learning at scale. However, low-resource problems are under-explored in computer vision. In this paper, we address this gap and explore the challenges of low-resource image tasks with vision foundation models. We first collect a benchmark of genuinely low-resource image data, covering historic maps, circuit diagrams, and mechanical drawings. These low-resource settings all share three challenges: data scarcity, fine-grained differences, and the distribution shift from natural images to the specialized domain of interest. While existing foundation models have shown impressive generalizability, we find they cannot transfer well to our low-resource tasks. To begin to tackle the challenges of low-resource vision, we introduce one simple baseline per challenge. Specifically, we i) enlarge the data space by generative models, ii) adopt the best sub-kernels to encode local regions for fine-grained difference discovery and iii) learn attention for specialized domains. Experiments on our three low-resource tasks demonstrate our proposals already provide a better baseline than transfer learning, data augmentation, and fine-grained methods. This highlights the unique characteristics and challenges of low-resource vision for foundation models that warrant further investigation. Project page: https://xiaobai1217.github.io/Low-Resource-Vision/.
SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality
Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人中心）；Princeton University（普林斯顿大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality
Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港中文大学人工智能机器人中心）；Princeton University（普林斯顿大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
SatelliteCalculator: A Multi-Task Vision Foundation Model for Quantitative Remote Sensing Inversion
Faculty of Computer Science and Information Technology, Universiti Malaya（马来亚大学计算机科学与信息技术学院）；Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）
ZeroSlide: Is Zero-Shot Classification Adequate for Lifelong Learning in Whole-Slide Image Analysis in the Era of Pathology Vision-Language Foundation Models?
Nara Institute of Science and Technology（奈良科学技术研究所）
Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation
School of Artificial Intelligence, Xidian University, China（西安电子科技大学人工智能学院）；Department of Information Engineering and Computer Science, University of Trento, Italy（特伦托大学信息工程与计算机科学系）；Department of Automation, Tsinghua University, China（清华大学自动化系）；School of Computer Science and Information Engineering, Hefei University of Technology, China（合肥工业大学计算机科学与信息工程学院）
BioVFM-21M: Benchmarking and Scaling Self-Supervised Vision Foundation Models for Biomedical Image Analysis
Paul C. Lauterbur Research Center for Biomedical Imaging（生物医学成像保罗·劳特伯研究中心）；Shenzhen Institutes of Advanced Technology（深圳先进技术研究院）；Chinese Academy of Sciences（中国科学院）；Pengcheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Faculty of Applied Sciences（应用科学学院）
UltraBench 2: Towards Robust Evaluation of Vision Foundation Models on Ultrasound
University of California, Los Angeles（加州大学洛杉矶分校）；Université Laval（拉瓦尔大学）；Mila – Quebec AI Institute（米拉-魁北克人工智能研究所）；UCLA David Geffen School of Medicine（加州大学洛杉矶分校大卫·格芬医学院）
Cross-Domain Foundation Model Adaptation: Pioneering Computer Vision Models for Geophysical Data Analysis
School of Earth and Space Sciences, University of Science and Technology of China（中国科学技术大学地球和空间科学学院）；Microsoft Applied Sciences Group（微软应用科学组）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced with Clinical Knowledge
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Washington（华盛顿大学）；Shenzhen Institutes of Advanced Technology（深圳先进技术研究所）；Chinese Academy of Sciences（中国科学院）；State Key Laboratory of Ophthalmology（眼科学国家重点实验室）；Zhongshan Ophthalmic Center（中山眼科中心）；Sun Yat-sen University（中山大学）；Guangdong Provincial Key Laboratory of Ophthalmology and Visual Science（广东省眼科学与视觉科学重点实验室）；Guangdong Provincial Clinical Research Center for Ocular Diseases（广东省眼科临床研究中心）；Department of Bioengineering（生物工程系）；Department of Radiology（放射科）
VisionGPT: Vision-Language Understanding Agent Using Generalized Multimodal Framework
Stanford University（斯坦福大学）；Peking University（北京大学）；Harvard University（哈佛大学）；University of Washington, Seattle（华盛顿大学西雅图分校）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks
Nanjing University（南京大学）；The University of Hong Kong（香港大学）；OpenGVLab, Shanghai AI Laboratory（上海人工智能实验室 OpenGVLab）；The Chinese University of Hong Kong（香港中文大学）；University of Science and Technology of China（中国科学技术大学）；Tsinghua University（清华大学）；SenseTime Research（商汤科技研究院）
Fine-tuning vision foundation model for crack segmentation in civil infrastructures
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Department of Civil Engineering, Tsinghua University（清华大学土木工程系）；School of Civil and Resource Engineering, University of Science and Technology Beijing（北京科技大学土木与资源工程学院）
SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models
Max Planck Institute for Informatics（马克斯·普朗克研究所信息学研究所）；Saarland Informatics Campus（萨尔州信息学校园）；CISPA Helmholtz Center for Information Security（信息安全霍夫曼中心）；University of Freiburg（弗赖堡大学）
Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniques
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Peng Cheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；The University of Hong Kong（香港大学）；Pazhou Laboratory (Huangpu)（琶洲实验室（黄埔））
