Segment Anything Model (SAM) 是一个最近开发的大型模型，用于计算机视觉任务中的通用分割。SAM 使用 1100 万张图像和超过 10 亿个掩码进行训练，能够为自然场景图像中的各种物体生成分割结果。SAM 可以被视为一个通用的分割感知模型（将图像划分为语义上有意义的区域）。因此，如何利用这样的大型基础模型进行医学图像分割是一个新兴的研究目标。本文表明，尽管 SAM 不能立即为医学图像数据提供高质量的分割，但其生成的掩码、特征和稳定性分数对于构建和训练更好的医学图像分割模型是有用的。特别是，我们演示了如何使用 SAM 来增强常用医学图像分割模型（如 U-Net）的图像输入。在三个分割任务上的实验证明了我们提出的 SAMAug 方法的有效性。代码可在 \u0060https://github.com/yizhezhang2000/SAMAug\u0060 获取。
英文摘要：
The Segment Anything Model (SAM) is a recently developed large model for general-purpose segmentation for computer vision tasks. SAM was trained using 11 million images with over 1 billion masks and can produce segmentation results for a wide range of objects in natural scene images. SAM can be viewed as a general perception model for segmentation (partitioning images into semantically meaningful regions). Thus, how to utilize such a large foundation model for medical image segmentation is an emerging research target. This paper shows that although SAM does not immediately give high-quality segmentation for medical image data, its generated masks, features, and stability scores are useful for building and training better medical image segmentation models. In particular, we demonstrate how to use SAM to augment image input for commonly-used medical image segmentation models (e.g., U-Net). Experiments on three segmentation tasks show the effectiveness of our proposed SAMAug method. The code is available at \url{https://github.com/yizhezhang2000/SAMAug}.
South China University of Technology（华南理工大学）；University of Utah（犹他大学）
We introduce the idea of inter-slice image augmentation whereby the numbers of the medical images and the corresponding segmentation labels are increased between two consecutive images in order to boost medical image segmentation accuracy. Unlike conventional data augmentation methods in medical imaging, which only increase the number of training samples directly by adding new virtual samples using simple parameterized transformations such as rotation, flipping, scaling, etc., we aim to augment data based on the relationship between two consecutive images, which increases not only the number b…
Boosting Semi-Supervised 2D Human Pose Estimation by Revisiting Data Augmentation and Consistency Training
School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；Chongqing Academy of Science and Technology（重庆科学技术研究院）；Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系）
As more and more artificial intelligence (AI) technologies move from the laboratory to real-world applications, the open-set and robustness challenges brought by data from the real world have received increasing attention. Data augmentation is a widely used method to improve model performance, and some recent works have also confirmed its positive effect on the robustness of AI models. However, most of the existing data augmentation methods are heuristic, lacking the exploration of their internal mechanisms. We apply the explainable artificial intelligence (XAI) method, explore the internal me…
Data61-CSIRO（Data61-联邦科学与工业研究组织）；Australian National University（澳大利亚国立大学）；University of Adelaide（阿德莱德大学）
We present You Only Cut Once (YOCO) for performing data augmentations. YOCO cuts one image into two pieces and performs data augmentations individually within each piece. Applying YOCO improves the diversity of the augmentation per sample and encourages neural networks to recognize objects from partial information. YOCO enjoys the properties of parameter-free, easy usage, and boosting almost all augmentations for free. Thorough experiments are conducted to evaluate its effectiveness. We first demonstrate that YOCO can be seamlessly applied to varying data augmentations, neural network architec…
Zhejiang University（浙江大学）；Beijing Aircraft Technology Research Institute（北京飞机技术研究院）；Shanghai AI Laboratory（上海人工智能实验室）
Current 3D object detection methods heavily rely on an enormous amount of annotations. Semi-supervised learning can be used to alleviate this issue. Previous semi-supervised 3D object detection methods directly follow the practice of fully-supervised methods to augment labeled and unlabeled data, which is sub-optimal. In this paper, we design a data augmentation method for semi-supervised learning, which we call Semi-Sampling. Specifically, we use ground truth labels and pseudo labels to crop gt samples and pseudo samples on labeled frames and unlabeled frames, respectively. Then we can genera…
Technical Report for ICRA 2025 GOOSE 2D Semantic Segmentation Challenge: Boosting Off-Road Segmentation via Photometric Distortion and Exponential Moving Average
Daegu-Gyeongbuk Research Center, Electronics and Telecommunications Research Institute (ETRI)（大邱-庆尚研究中心，电子电信研究所）
Boosting Overlapping Organoid Instance Segmentation Using Pseudo-Label Unmixing and Synthesis-Assisted Learning
School of Biomedical Engineering, Sun Yat-sen University（中山大学生物医学工程学院）；Peng Cheng Laboratory（鹏城实验室）；The First Affiliated Hospital of Shenzhen University, Shenzhen Second People's Hospital（深圳大学第一附属医院、深圳第二人民医院）；The Research Center of Clinical Medicine, Affiliated Hospital of Nantong University, Medical School of Nantong University（临床医学研究中心、南通大学附属医院、南通大学医学院）；Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences（慈溪生物医学工程研究所、宁波材料技术与工程研究所、中国科学院）
Instance segmentation requires a large number of training samples to achieve satisfactory performance and benefits from proper data augmentation. To enlarge the training set and increase the diversity, previous methods have investigated using data annotation from other domain (e.g. bbox, point) in a weakly supervised mechanism. In this paper, we present a simple, efficient and effective method to augment the training set using the existing instance mask annotations. Exploiting the pixel redundancy of the background, we are able to improve the performance of Mask R-CNN for 1.7 mAP on COCO datas…
Complicated underwater environments bring new challenges to object detection, such as unbalanced light conditions, low contrast, occlusion, and mimicry of aquatic organisms. Under these circumstances, the objects captured by the underwater camera will become vague, and the generic detectors often fail on these vague objects. This work aims to solve the problem from two perspectives: uncertainty modeling and hard example mining. We propose a two-stage underwater detector named boosting R-CNN, which comprises three key components. First, a new region proposal network named RetinaRPN is proposed,…
While efficient architectures and a plethora of augmentations for end-to-end image classification tasks have been suggested and heavily investigated, state-of-the-art techniques for audio classifications still rely on numerous representations of the audio signal together with large architectures, fine-tuned from large datasets. By utilizing the inherited lightweight nature of audio and novel audio augmentations, we were able to present an efficient end-to-end network with strong generalization ability. Experiments on a variety of sound classification sets demonstrate the effectiveness and robu…
Edith Cowan University（伊迪斯科文大学）；University of Western Australia（西澳大学）
COVID-19 classification using chest Computed Tomography (CT) has been found pragmatically useful by several studies. Due to the lack of annotated samples, these studies recommend transfer learning and explore the choices of pre-trained models and data augmentation. However, it is still unknown if there are better strategies than vanilla transfer learning for more accurate COVID-19 classification with limited CT data. This paper provides an affirmative answer, devising a novel `model' augmentation technique that allows a considerable performance boost to transfer learning for the task. Our meth…
The University of Adelaide（阿德莱德大学）；Adobe Inc.（Adobe公司）；Zhejiang University（浙江大学）
Deep image matting methods have achieved increasingly better results on benchmarks (e.g., Composition-1k/alphamatting.com). However, the robustness, including robustness to trimaps and generalization to images from different domains, is still under-explored. Although some works propose to either refine the trimaps or adapt the algorithms to real-world images via extra data augmentation, none of them has taken both into consideration, not to mention the significant performance deterioration on benchmarks while using those data augmentation. To fill this gap, we propose an image matting method w…
Delta-K: Boosting Multi-Instance Generation via Cross-Attention Augmentation
School of Software Engineering, Sun Yat-sen University（中山大学软件学院）；Nanjing University（南京大学）；College of Management and Economics, Tianjin University（天津大学管理学院）
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；School of Engineering, Newcastle University（纽卡斯尔大学工程学院）
The recent advance in deep generative models outlines a promising perspective in the realm of Zero-Shot Learning (ZSL). Most generative ZSL methods use category semantic attributes plus a Gaussian noise to generate visual features. After generating unseen samples, this family of approaches effectively transforms the ZSL problem into a supervised classification scheme. However, the existing models use a single semantic attribute, which contains the complete attribute information of the category. The generated data also carry the complete attribute information, but in reality, visual samples usu…
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
Continuous sign language recognition (SLR) deals with unaligned video-text pair and uses the word error rate (WER), i.e., edit distance, as the main evaluation metric. Since it is not differentiable, we usually instead optimize the learning model with the connectionist temporal classification (CTC) objective loss, which maximizes the posterior probability over the sequential alignment. Due to the optimization gap, the predicted sentence with the highest decoding probability may not be the best choice under the WER metric. To tackle this issue, we propose a novel architecture with cross modalit…
Boosting Classification with Quantum-Inspired Augmentations
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；RPTU Kaiserslautern-Landau（科隆-兰道大学）；Department of Computer Science and Research Initiative QC-AI（计算机科学系和量子人工智能研究计划）；Department of Physics（物理系）
Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Prompts
Shanghai Artificial Intelligence Lab（上海人工智能实验室）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of California Santa Cruz（加利福尼亚大学圣克鲁兹分校）；West China Biomedical Big Data Center, Sichuan University West China Hospital（四川大学华西医院华西生物医学大数据中心）
Bosch Center for Artificial Intelligence（博世人工智能中心）
Despite data augmentation being a de facto technique for boosting the performance of deep neural networks, little attention has been paid to developing augmentation strategies for generative adversarial networks (GANs). To this end, we introduce a novel augmentation scheme designed specifically for GAN-based semantic image synthesis models. We propose to randomly warp object shapes in the semantic label maps used as an input to the generator. The local shape discrepancies between the warped and non-warped label maps and images enable the GAN to learn better the structural and geometric details…
Boosting Adversarial Transferability across Model Genus by Deformation-Constrained Warping
Shenzhen University（深圳大学）；Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳人工智能与机器人学会）；Ghent University（根特大学）；University of Leicester（莱斯特大学）
Having access to accurate game state information is of utmost importance for any artificial intelligence task including game-playing, testing, player modeling, and procedural content generation. Self-Supervised Learning (SSL) techniques have shown to be capable of inferring accurate game state information from the high-dimensional pixel input of game footage into compressed latent representations. Contrastive Learning is a popular SSL paradigm where the visual understanding of the game's images comes from contrasting dissimilar and similar game states defined by simple image augmentation metho…
Vicomtech Foundation, Basque Research and Technology Alliance (BRTA)（维康科技基金会，巴斯克研究与技术联盟（BRTA））；University of the Basque Country（巴斯克大学）
In this paper, we address the problem of face recognition with masks. Given the global health crisis caused by COVID-19, mouth and nose-covering masks have become an essential everyday-clothing-accessory. This sanitary measure has put the state-of-the-art face recognition models on the ropes since they have not been designed to work with masked faces. In addition, the need has arisen for applications capable of detecting whether the subjects are wearing masks to control the spread of the virus. To overcome these problems a full training pipeline is presented based on the ArcFace work, with sev…
Improving the Transferability of Adversarial Examples by Feature Augmentation
Zhejiang University（浙江大学）；Defense Innovation Institute, Chinese Academy of Military Science（中国军事科学院国防创新研究院）；Intelligent Game and Decision Laboratory（智能博弈与决策实验室）
Mississippi State University（密西西比州立大学）；Michigan State University（密歇根州立大学）；United States Department of Agriculture, Agricultural Research Service（美国农业部农业研究局）
In agricultural image analysis, optimal model performance is keenly pursued for better fulfilling visual recognition tasks (e.g., image classification, segmentation, object detection and localization), in the presence of challenges with biological variability and unstructured environments. Large-scale, balanced and ground-truthed image datasets, however, are often difficult to obtain to fuel the development of advanced, high-performance models. As artificial intelligence through deep learning is impacting analysis and modeling of agricultural images, data augmentation plays a crucial role in b…
Image to Pseudo-Episode: Boosting Few-Shot Segmentation by Unlabeled Data
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Intelligent Computing Technology, CAS（中国科学院智能计算技术研究所）；Microsoft Research Asia（微软亚洲研究院）
PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；School of Information Science and Technology, ShanghaiTech University（上海科技大学信息科学与技术学院）；OPPO（OPPO公司）
The University of Adelaide（阿德莱德大学）；The University of Sydney（悉尼大学）
The challenge of fine-grained visual recognition often lies in discovering the key discriminative regions. While such regions can be automatically identified from a large-scale labeled dataset, a similar method might become less effective when only a few annotations are available. In low data regimes, a network often struggles to choose the correct regions for recognition and tends to overfit spurious correlated patterns from the training data. To tackle this issue, this paper proposes the self-boosting attention mechanism, a novel method for regularizing the network to focus on the key region…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）
We present a research study aimed at testing of applicability of machine learning techniques for prediction of permeability of digitized rock samples. We prepare a training set containing 3D images of sandstone samples imaged with X-ray microtomography and corresponding permeability values simulated with Pore Network approach. We also use Minkowski functionals and Deep Learning-based descriptors of 3D images and 2D slices as input features for predictive model training and prediction. We compare predictive power of various feature sets and methods. The later include Gradient Boosting and vario…
