Our objective in this work is video-text retrieval - in particular a joint embedding that enables efficient text-to-video retrieval. The challenges in this area include the design of the visual architecture and the nature of the training data, in that the available large scale video-text training datasets, such as HowTo100M, are noisy and hence competitive performance is achieved only at scale through large amounts of compute. We address both these challenges in this paper. We propose an end-to-end trainable model that is designed to take advantage of both large-scale image and video captioning datasets. Our model is an adaptation and extension of the recent ViT and Timesformer architectures, and consists of attention in both space and time. The model is flexible and can be trained on both image and video text datasets, either independently or in conjunction. It is trained with a curriculum learning schedule that begins by treating images as 'frozen' snapshots of video, and then gradually learns to attend to increasing temporal context when trained on video datasets. We also provide a new video-text pretraining dataset WebVid-2M, comprised of over two million videos with weak captions scraped from the internet. Despite training on datasets that are an order of magnitude smaller, we show that this approach yields state-of-the-art results on standard downstream video-retrieval benchmarks including MSR-VTT, MSVD, DiDeMo and LSMDC.
DINOv3 as a Frozen Encoder for CRPS-Oriented Probabilistic Rainfall Nowcasting
Advanced Data Analysis and Management, University of Feira de Santana（费拉德桑塔大学高级数据分析与管理中心）；National Institute of Metrology, Quality and Technology（计量与技术质量国家研究院）；Georgia Institute of Technology（佐治亚理工学院）；University of West Bohemia in Pilsen（比尔森大学）
Big, Bright, or Invisible: A Frozen-Feature Benchmark of 3D CT Foundation Models
Technical University of Munich (TUM)（慕尼黑工业大学（TUM））；TUM University Hospital（慕尼黑工业大学医院）；Imperial College London（伦敦帝国理工学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；UKE Hamburg（汉堡大学医院）
Fixed-Budget Parameter-Efficient Training with Frozen Encoders Improves Multimodal Chest X-Ray Classification
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, India（计算机科学与工程系，印度理工学院Kharagpur分校）；Knight Foundation School of Computing and Information Sciences, Florida International University, Florida, USA（骑士基金会计算与信息科学学院，佛罗里达国际大学）
How to Make Cross Encoder a Good Teacher for Efficient Image-Text Retrieval?
State Key Laboratory of Multimodal Artificial Intelligence Systems（多模态人工智能系统国家重点实验室）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；The University of Hong Kong（香港大学）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders
King Abdullah University of Science and Technology（国王阿卜杜勒-阿齐兹大学科学与技术大学）；Visual Recognition Group（视觉识别组）；Faculty of Electrical Engineering（电气工程学院）；Czech Technical University in Prague（布拉格捷克技术大学）
HyperTopo-Adapters: Geometry- and Topology-Aware Segmentation of Leaf Lesions on Frozen Encoders
Missouri Maize Computation and Vision Laboratory (MMCV)（密苏里州玉米计算与视觉实验室）；Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）；University of Missouri（密苏里大学）
SEDS: Semantically Enhanced Dual-Stream Encoder for Sign Language Retrieval
University of Science and Technology of China（中国科学技术大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；Merchants Union Consumer Finance Company Limited（招商局消费金融有限公司）
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval
The University of Hong Kong（香港大学）；ARC Lab（ARC实验室）；UC Berkeley（加州大学伯克利分校）；National University of Singapore（新加坡国立大学）；Tsinghua University（清华大学）；Tencent PCG（腾讯平台与内容事业群）
FusionBERT: Multi-View Image--3D Retrieval via Cross-Attention Visual Fusion and Normal-Aware 3D Encoder
IROOTECH TECHNOLOGY；Wolf 1069 b Lab, Sany Group（三一集团Wolf 1069 b实验室）；Zhejiang University（浙江大学）；School of Computer Science and Engineering, Central South University（中南大学计算机科学与工程学院）；BPIT, Sany Group（三一集团BPIT）
FLAME: Frozen Large Language Models Enable Data-Efficient Language-Image Pre-training
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；Shenzhen University of Advanced Technology（深圳先进技术大学）；Guangdong Provincial Key Laboratory of Computility Microelectronics（广东省计算微电子重点实验室）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）
SEA: Sentence Encoder Assembly for Video Retrieval by Textual Queries
Renmin University of China（中国人民大学）
通过文本查询来检索无标签视频，称为Ad-hoc Video Search（AVS），是多媒体数据管理和检索的核心主题。AVS的成功依赖于跨模态表示学习，该学习将查询语句和视频编码到公共空间中以进行语义相似度计算。受先前少数作品在结合多个句子编码器方面取得的初步成功的启发，本文通过开发一种新的通用方法来有效地利用不同的句子编码器，向前迈出了一步。所提出的方法，我们术语句子编码器组件（SEA）的新颖性，是双重的。首先，与仅使用单个公共空间的现有技术不同，SEA支持多个编码器特定的公共空间中的文本-视频匹配。这样的属性防止匹配被产生比其他编码器长得多的编码向量的特定编码器支配。其次，为了探索个体公共空间之间的互补性，我们提出了多空间多损失学习。在MSR-VTT、TRECVID AVS 2016-2019、TGIF和MSVD四个基准测试上的大量实验表明，SEA超越了最先进的水平。此外，SEA非常容易实现。所有这些都使SEA成为AVS的一个有吸引力的解决方案，并有希望通过收获新的句子编码器来不断推进任务。
Recognizing Families through Images with Pretrained Encoder
Faculty of Information Technology, University of Science, VNU-HCM（越南国家大学胡志明市分校科学大学信息技术学院）
亲属关系验证和亲属关系检索是计算机视觉中的新兴任务。亲属关系验证的目的是确定两个人脸图像是否来自相关的人，而亲属关系检索是从图像库中检索可能与一个人相关的人脸图像的任务。他们引入了独特的挑战，因为隐藏的关系和功能，进行面部图像之间的固有特性。我们采用3种方法，FaceNet，Siamese VGG-Face以及FaceNet和VGG-Face模型的组合作为特征提取器，在2020年的Recognizing Family in The Wild比赛中获得了第9名的亲属关系验证和第5名的亲属关系检索。然后，我们使用StyleGAN 2作为另一个编码器进行了进一步的实验，结果没有任何改善。
