This paper examines performance evaluation criteria for basic vision tasks involving sets of objects namely, object detection, instance-level segmentation and multi-object tracking. The rankings of algorithms by an existing criterion can fluctuate with different choices of parameters, e.g. Intersection over Union (IoU) threshold, making their evaluations unreliable. More importantly, there is no means to verify whether we can trust the evaluations of a criterion. This work suggests a notion of trustworthiness for performance criteria, which requires (i) robustness to parameters for reliability, (ii) contextual meaningfulness in sanity tests, and (iii) consistency with mathematical requirements such as the metric properties. We observe that these requirements were overlooked by many widely-used criteria, and explore alternative criteria using metrics for sets of shapes. We also assess all these criteria based on the suggested requirements for trustworthiness.
YOLO with Kolmogorov-Arnold networks and vision-language foundation models for interpretable object detection with trustworthy multimodal AI in computer vision perception
University of Bath（巴斯大学）；Zhejiang University（浙江大学）
MedVIGIL: Evaluating Trustworthy Medical VLMs Under Broken Visual Evidence
University of Georgia（佐治亚大学）；Harvard Medical School（哈佛医学院）；Chungbuk National University（Chungbuk国立大学）；Chungnam National University Hospital（Chungnam国立大学医院）；National University of Singapore（新加坡国立大学）；New York University（纽约大学）；University of Sydney（悉尼大学）；New Jersey Institute of Technology（新泽西理工学院）
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）
Building AI models with trustworthiness is important especially in regulated areas such as healthcare. In tackling COVID-19, previous work uses convolutional neural networks as the backbone architecture, which has shown to be prone to over-caution and overconfidence in making decisions, rendering them less trustworthy -- a crucial flaw in the context of medical imaging. In this study, we propose a feature learning approach using Vision Transformers, which use an attention-based mechanism, and examine the representation learning capability of Transformers as a new backbone architecture for medi…
University of Waterloo（滑铁卢大学）；National Research Council Canada（加拿大国家研究委员会）
Computer vision and machine learning are playing an increasingly important role in computer-assisted diagnosis; however, the application of deep learning to medical imaging has challenges in data availability and data imbalance, and it is especially important that models for medical imaging are built to be trustworthy. Therefore, we propose TRUDLMIA, a trustworthy deep learning framework for medical image analysis, which adopts a modular design, leverages self-supervised pre-training, and utilizes a novel surrogate loss function. Experimental evaluations indicate that models generated from the…
HiLRP: Toward One Trustworthy Explanation for Vision Transformer: Conservation-Valid Attribution via Attention Primitives
Faculty of Engineering, University of Peradeniya（佩拉德尼亚大学工程学院）；School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算技术学院）；School of Engineering & Digital Technologies, University of Southern Queensland（南昆士兰大学工程与数字技术学院）；School of Engineering & Technologies, UNSW（新南威尔士大学工程与技术学院）；Faculty of Science and Technology, Charles Darwin University（查尔斯达尔文大学科学技术学院）
With the maturing of deep learning systems, trustworthiness is becoming increasingly important for model assessment. We understand trustworthiness as the combination of explainability and robustness. Generative classifiers (GCs) are a promising class of models that are said to naturally accomplish these qualities. However, this has mostly been demonstrated on simple datasets such as MNIST and CIFAR in the past. In this work, we firstly develop an architecture and training scheme that allows GCs to operate on a more relevant level of complexity for practical computer vision, namely the ImageNet…
What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning
Birla Institute of Technology and Science, Pilani（贝拉理工科学学院皮拉尼分校）；University of California, Merced（加州大学默塞德分校）
This draft summarizes some basics about geometric computer vision needed to implement efficient computer vision algorithms for applications that use measurements from at least one digital camera mounted on a moving platform with a special focus on automotive applications processing image streams taken from cameras mounted on a car. Our intention is twofold: On the one hand, we would like to introduce well-known basic geometric relations in a compact way that can also be found in lecture books about geometric computer vision like [1, 2]. On the other hand, we would like to share some experience…
Evaluating Self-Correcting Vision Agents Through Quantitative and Qualitative Metrics
The Thørväld Group（Thørväld集团）；Inria Paris-Rocquencourt（巴黎-罗克琴堡Inria）；Rajiv Gandhi University Doimukh（拉贾·甘地大学多伊穆克）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒姆研究实验室）；The Kumquat Consortium（昆夸特联盟）
This paper surveys vision-language pre-training (VLP) methods for multimodal intelligence that have been developed in the last few years. We group these approaches into three categories: ($i$) VLP for image-text tasks, such as image captioning, image-text retrieval, visual question answering, and visual grounding; ($ii$) VLP for core computer vision tasks, such as (open-set) image classification, object detection, and segmentation; and ($iii$) VLP for video-text tasks, such as video captioning, video-text retrieval, and video question answering. For each category, we present a comprehensive re…
Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning
McGill University（麦吉尔大学）；Massachusetts Institute of Technology（麻省理工学院）；Shanghai Jiao Tong University（上海交通大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Florida（佛罗里达大学）；The University of Hong Kong（香港大学）
RISE Research Institutes of Sweden（瑞典RISE研究所）；Lund University（隆德大学）
Machine Learning (ML) is a fundamental part of modern perception systems. In the last decade, the performance of computer vision using trained deep neural networks has outperformed previous approaches based on careful feature engineering. However, the opaqueness of large ML models is a substantial impediment for critical applications such as in the automotive context. As a remedy, Gradient-weighted Class Activation Mapping (Grad-CAM) has been proposed to provide visual explanations of model internals. In this paper, we demonstrate how Grad-CAM heatmaps can be used to increase the explainabilit…
Object detection is a very important basic research direction in the field of computer vision and a basic method for other advanced tasks in the field of computer vision. It has been widely used in practical applications such as object tracking, video behavior recognition and underwater robotics vision. The Cascade-RCNN and Deformable Convolution Network are both classical and excellent object detection algorithms. In this report, we evaluate our Cascade-DCN based method on underwater optical image and acoustics image datasets with different engineering tricks and augumentation.
This article discusses how concepts and methods of complex networks can be applied to real-time imaging and computer vision. After a brief introduction of complex networks basic concepts, their use as means to represent and characterize images, as well as for modeling visual saliency, are briefly described. The possibility to apply complex networks in order to model and simulate the performance of parallel and distributed computing systems for performance of visual methods is also proposed.
A basic assumption of statistical learning theory is that train and test data are drawn from the same underlying distribution. Unfortunately, this assumption doesn't hold in many applications. Instead, ample labeled data might exist in a particular `source' domain while inference is needed in another, `target' domain. Domain adaptation methods leverage labeled data from both domains to improve classification on unseen data in the target domain. In this work we survey domain transfer learning methods for various application domains with focus on recent work in Computer Vision.
