Out-of-domain (OOD) detection is a crucial component in industrial applications as it helps identify when a model encounters inputs that are outside the training distribution. Most industrial pipelines rely on pre-trained models for downstream tasks such as CNN or Vision Transformers. This paper investigates the performance of those models on the task of out-of-domain detection. Our experiments demonstrate that pre-trained transformers models achieve higher detection performance out of the box. Furthermore, we show that pre-trained ViT and CNNs can be combined with refinement methods such as CIDER to improve their OOD detection performance even more. Our results suggest that transformers are a promising approach for OOD detection and set a stronger baseline for this task in many contexts
GIT: A Generative Image-to-text Transformer for Vision and Language
Microsoft Cloud and AI(微软云与人工智能部门)
在本文中,我们设计和训练一个生成的图像到文本的Transformer,GIT,统一的视觉语言任务,如图像/视频描述和问答。虽然生成模型在预训练和微调之间提供了一致的网络架构,但现有的工作通常包含复杂的结构(单/多模态编码器/解码器),并依赖于外部模块,如对象检测器/标签和光学字符识别(OCR)。在GIT中,我们将架构简化为单一语言建模任务下的一个图像编码器和一个文本解码器。我们还扩展了预训练数据和模型大小,以提高模型性能。没有花里胡哨,我们的GIT在12个具有挑战性的基准上建立了新的艺术状态。例如,我们的模型在TextCaps上首次超过了人类的表现(138.2 vs. CIDEr中的125.5)。此外,我们提出了一种新的方案,基于生成的图像分类和场景文本识别,在标准基准测试中取得了不错的性能。代码在\url{https://github.com/microsoft/GenerativeImage2Text}发布。
Combining Transformer Generators with Convolutional Discriminators
Fraunhofer ITWM(弗劳恩霍夫工业数学研究所);University of Heidelberg(海德堡大学);Fraunhofer Center Machine Learning(弗劳恩霍夫机器学习中心);Technical University of Kaiserslautern(凯泽斯劳滕工业大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Offenburg University(奥芬堡大学)
Combining Microscopy Data and Metadata for Reconstruction of Cellular Traction Forces Using a Hybrid Vision Transformer-U-Net
Fakultät Physik, Technische Universität Dortmund(物理系,德意志理工大学多特蒙德分校);Faculty of Physics and Center for NanoScience, Ludwig-Maximilians-Universität München(物理系和纳米科学中心,路德维希-马克西米利安大学慕尼黑分校);Department of Veterinary Sciences, Ludwig-Maximilians-Universität München(兽医学系,路德维希-马克西米利安大学慕尼黑分校)
T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Image
School of Data Science, Fudan University(复旦大学数据科学学院);Youtu Lab, Tencent(腾讯优图实验室);School of Intelligence Science and Technology, Nanjing University(南京大学智能科学与技术学院);Google(谷歌公司)
ConViViT -- A Deep Neural Network Combining Convolutions and Factorized Self-Attention for Human Activity Recognition
Efrei Research Lab(Efrei研究实验室);Paris Pantheon-Assas University(巴黎先贤祠-阿萨斯大学);Paris Panthéon-Assas University(巴黎先贤祠-阿萨斯大学);ICube UMR 7357, CNRS(法国国家科学研究中心ICube UMR 7357实验室);Université de Strasbourg(斯特拉斯堡大学);Caplogy SAS(Caplogy公司)