PaLI-3 Vision Language Models: Smaller, Faster, Stronger
Google Research(谷歌研究院);Google DeepMind(谷歌DeepMind);Google Cloud(谷歌云)
Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov, Jialin Wu, Paul Voigtlaender, Basil Mustafa, Sebastian Goodman, Ibrahim Alabdulmohsin, Piotr Padlewski, D… 展开作者
Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov, Jialin Wu, Paul Voigtlaender, Basil Mustafa, Sebastian Goodman, Ibrahim Alabdulmohsin, Piotr Padlewski, Daniel Salz, Xi Xiong, Daniel Vlasic, Filip Pavetic, Keran Rong, Tianli Yu, Daniel Keysers, Xiaohua Zhai, Radu Soricut
This paper presents PaLI-3, a smaller, faster, and stronger vision language model (VLM) that compares favorably to similar models that are 10x larger. As part of arriving at this strong performance, we compare Vision Transformer (ViT) models pretrained using classification objectives to contrastively (SigLIP) pretrained ones. We find that, while slightly underperforming on standard image classification benchmarks, SigLIP-based PaLI shows superior performance across various multimodal benchmarks, especially on localization and visually-situated text understanding. We scale the SigLIP image encoder up to 2 billion parameters, and achieves a new state-of-the-art on multilingual cross-modal retrieval. We hope that PaLI-3, at only 5B parameters, rekindles research on fundamental pieces of complex VLMs, and could fuel a new generation of scaled-up models.
Constructing Stronger and Faster Baselines for Skeleton-based Action Recognition
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shandong University of Science and Technology(山东科技大学);Artificial Intelligence Research, Chinese Academy of Sciences(中国科学院人工智能研究院)
A Faster, Lighter and Stronger Deep Learning-Based Approach for Place Recognition
Xiamen University(厦门大学)
视觉位置识别是摄像机定位和环路闭合检测系统的重要组成部分,在计算机视觉、机器人和AR/VR等多个领域引起了广泛的兴趣。在这项工作中,我们提出了一种更快,更轻,更强大的方法,可以生成具有更少参数的模型,并且可以在推理阶段花费更少的时间。我们设计了RepVGG-lite作为我们的架构中的骨干网络,它比其他一般网络在地点识别任务中更具鉴别力。RepVGG-lite在实现更高性能的同时具有更多的速度优势。在特征提取阶段,我们只从全局描述符中提取一个尺度的块级描述符。然后,我们设计了一个可训练的特征匹配器,利用的空间关系的功能和他们的视觉外观,这是基于注意力机制。在具有挑战性的基准数据集上进行的综合实验表明,所提出的方法优于最近其他最先进的学习方法,并实现了更高的推理速度。我们的系统比Patch-NetVLAD少14倍的参数,低6.8倍的理论FLOPs,并且在特征提取和特征匹配中运行速度快21和33倍。此外,我们的方法的性能是0.5\%,比补丁NetVLAD在Recall@1。我们使用Mapillary Street Level Sequences数据集的子集对所有其他具有挑战性的条件进行实验。
Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition
University of Chinese Academy of Sciences(中国科学院大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);College of Electrical Engineering and Automation, Shandong University of Science and Technology(山东科技大学电气工程与自动化学院);School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院)
Lumina-Next: Making Lumina-T2X Stronger and Faster with Next-DiT
Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Beijing University of Posts and Telecommunications(北京邮电大学);Beihang University(北京航空航天大学);Nanyang Technological University(南洋理工大学)