We present Xray-Visual, a unified vision model architecture for large-scale image and video understanding trained on industry-scale social media data. Our model leverages over 15 billion curated image-text pairs and 10 billion video-hashtag pairs from Facebook and Instagram, employing robust data curation pipelines that incorporate balancing and noise suppression strategies to maximize semantic diversity while minimizing label noise. We introduce a three-stage training pipeline that combines self-supervised MAE, semi-supervised hashtag classification, and CLIP-style contrastive learning to jointly optimize image and video modalities. Our architecture builds on a Vision Transformer backbone enhanced with efficient token reorganization (EViT) for improved computational efficiency. Extensive experiments demonstrate that Xray-Visual achieves state-of-the-art performance across diverse benchmarks, including ImageNet for image classification, Kinetics and HMDB51 for video understanding, and MSCOCO for cross-modal retrieval. The model exhibits strong robustness to domain shift and adversarial perturbations. We further demonstrate that integrating large language models as text encoders (LLM2CLIP) significantly enhances retrieval performance and generalization capabilities, particularly in real-world environments. Xray-Visual establishes new benchmarks for scalable, multimodal vision models, while maintaining superior accuracy and computational efficiency.
Seeing Through Experts Eyes A Foundational Vision Language Model Trained on Radiologists Gaze and Reasoning
Bioengineering Department and Imperial-X(生物工程系和Imperial-X);Imperial College London(帝国理工学院伦敦分校);College of physics and information engineering, Fuzhou University(福州大学物理与信息工程学院);Department of Surgery and Cancer, Imperial College London(外科与癌症部门,帝国理工学院伦敦分校);National Heart and Lung Institute, Imperial College London(国家心脏和肺研究所,帝国理工学院伦敦分校);Cardiovascular Research Centre, Royal Brompton Hospital(心血管研究中心,皇家布里顿医院);School of Biomedical Engineering & Imaging Sciences, King’s College London(生物医学工程与成像科学学院,国王学院伦敦分校)
Promoting AI Equity in Science: Generalized Domain Prompt Learning for Accessible VLM Research
Shanghai Jiao Tong University(上海交通大学);Eastern Institute of Technology(东方理工大学);Yale University(耶鲁大学);Renmin University of China(中国人民大学);Southern University of Science and Technology(南方科技大学);Ningbo Institute of Digital Twin(宁波数字孪生研究院)
DEEVA: A Deep Learning and IoT Based Computer Vision System to Address Safety and Security of Production Sites in Energy Industry
当涉及到解决不同生产/施工现场的安全/安保相关需求时,准确检测工人、车辆、设备的存在非常重要,并构成了基于计算机视觉的监控系统(CVSS)的组成部分。传统的CVSS系统集中于使用不同的计算机视觉和模式识别算法,过度依赖于手动提取特征和小数据集,由于准确性低、需要专家知识和高计算成本而限制了它们的使用。本文的主要目标是为现场决策者提供实用而全面的基于深度学习和物联网的解决方案,以解决各种计算机视觉相关问题,如场景分类,场景中的目标检测,语义分割,我们的首要目标是解决这个网站正在发生什么以及它在哪里发生的中心问题,以自动化的方式最小化需要专门用于监视的人力资源。我们开发了Deep ExxonMobil Eye for Video Analysis(DEEVA)软件包,以分层方式处理场景分类、目标检测、语义分割和场景字幕。结果表明,使用RetinaNet对象检测器的迁移学习能够以高准确度(90%以上)检测工人、不同类型的车辆/施工设备、安全相关对象的存在。借助深度学习自动提取特征和物联网技术自动捕获,传输和处理大量实时图像,该框架是智能监控系统开发的重要一步,旨在解决安全/安全监控,生产力评估和未来决策领域的无数开放式问题。
RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture
Department of Computer Science and Engineering, Indian Institute of Technology Bombay(印度理工学院孟买分校计算机科学与工程系);Department of Biomedical Engineering, Johns Hopkins University(约翰霍普金斯大学生物医学工程系);Koita Centre for Digital Health, Indian Institute of Technology Bombay(印度理工学院孟买分校Koita数字健康中心)
STPNet: Scale-aware Text Prompt Network for Medical Image Segmentation
Xiamen University(厦门大学);University of Washington(华盛顿大学);UT Southwestern Medical Center(西南大学医学中心);University of Hull(赫尔大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering
The University of Texas at Arlington(德克萨斯大学阿灵顿分校);RIKEN(理化学研究所);The University of Tokyo(东京大学);National Institutes of Health Clinical Center(美国国立卫生研究院临床中心);National Cancer Center Research Institute(国立癌症中心研究所)