The goal of this paper is to improve the performance of pretrained Vision Transformer (ViT) models, particularly DINOv2, in image clustering task without requiring re-training or fine-tuning. As model size increases, high-norm artifacts anomaly appears in the patches of multi-head attention. We observe that this anomaly leads to reduced accuracy in zero-shot image clustering. These artifacts are characterized by disproportionately large values in the attention map compared to other patch tokens. To address these artifacts, we propose an approach called Inference-Time Attention Engineering (ITAE), which manipulates attention function during inference. Specifically, we identify the artifacts by investigating one of the Query-Key-Value (QKV) patches in the multi-head attention and attenuate their corresponding attention values inside the pretrained models. ITAE shows improved clustering accuracy on multiple datasets by exhibiting more expressive features in latent space. Our findings highlight the potential of ITAE as a practical solution for reducing artifacts in pretrained ViT models and improving model performance in clustering tasks without the need for re-training or fine-tuning.
Attenuation artifact detection and severity classification in intracoronary OCT using mixed image representations
Amsterdam UMC（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Amsterdam Cardiovascular Sciences（阿姆斯特丹心血管科学中心）；Radboud University Medical Center（拉德堡德大学医学中心）
PADM: A Physics-aware Diffusion Model for Attenuation Correction
AI4LIFE, Hanoi University of Science and Technology, Vietnam（AI4LIFE，越南河内科学技术大学）；Nagoya Univeristy, Japan（名古屋大学）；National Institute of Advanced Industrial Science and Technology, Japan（日本国家先进工业科学和技术研究院）；Military Central Hospital, Vietnam（越南108中央军医院）
Ultrasound imaging has been improving, but continues to suffer from inherent artifacts that are challenging to model, such as attenuation, shadowing, diffraction, speckle, etc. These artifacts can potentially confuse image analysis algorithms unless an attempt is made to assess the certainty of individual pixel values. Our novel confidence algorithms analyze pixel values using a directed acyclic graph based on acoustic physical properties of ultrasound imaging. We demonstrate unique capabilities of our approach and compare it against previous confidence-measurement algorithms for shadow-detect…
Lossy compression introduces complex compression artifacts, particularly the blocking artifacts, ringing effects and blurring. Existing algorithms either focus on removing blocking artifacts and produce blurred output, or restores sharpened images that are accompanied with ringing effects. Inspired by the deep convolutional networks (DCN) on super-resolution, we formulate a compact and efficient network for seamless attenuation of different compression artifacts. We also demonstrate that a deeper model can be effectively trained with the features learned in a shallow network. Following a simil…
Schepens Eye Research Institute of Massachusetts Eye and Ear（麻省眼耳医院谢本斯眼科研究所）；Harvard Medical School（哈佛医学院）；Eye and Vision Research Institute, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院眼与视觉研究所）；Massachusetts Eye and Ear（麻省眼耳医院）
Ophthalmic images and derivatives such as the retinal nerve fiber layer (RNFL) thickness map are crucial for detecting and monitoring ophthalmic diseases (e.g., glaucoma). For computer-aided diagnosis of eye diseases, the key technique is to automatically extract meaningful features from ophthalmic images that can reveal the biomarkers (e.g., RNFL thinning patterns) linked to functional vision loss. However, representation learning from ophthalmic images that links structural retinal damage with human vision loss is non-trivial mostly due to large anatomical variations between patients. The ta…
During orthopaedic surgery, the inserting of metallic implants or screws are often performed under mobile C-arm systems. Due to the high attenuation of metals, severe metal artifacts occur in 3D reconstructions, which degrade the image quality greatly. To reduce the artifacts, many metal artifact reduction algorithms have been developed and metal inpainting in projection domain is an essential step. In this work, a score-based generative model is trained on simulated knee projections and the inpainted image is obtained by removing the noise in conditional resampling process. The result implies…
Many images nowadays are captured from behind the glasses and may have certain stains discrepancy because of glass and must be processed to make differentiation between the glass and objects behind it. This research paper proposes an algorithm to remove the damaged or corrupted part of the image and make it consistent with other part of the image and to segment objects behind the glass. The damaged part is removed using total variation inpainting method and segmentation is done using kmeans clustering, anisotropic diffusion and watershed transformation. The final output is obtained by interpol…
CT-Less Attenuation Correction Using Multiview Ensemble Conditional Diffusion Model on High-Resolution Uncorrected PET Images
Department of Medical Imaging and Radiation Sciences（医学影像与放射科学系）；Université de Sherbrooke（Sherbrooke大学）；Sherbrooke Molecular Imaging Center（Sherbrooke分子影像中心）；CRCHUS；Nantes Université（Nantes大学）；Laboratoire CRCI2NA, INSERM, CNRS（CRCI2NA实验室，INSERM，CNRS）；Imaging Research and Technology (IR&T) Inc.（影像研究与技术公司）；Research Center on Aging, Department of Medicine（衰老研究中心，医学系）；Department of Electrical Engineering and Computer Engineering（电气工程与计算机工程系）
University of Michigan - Shanghai Jiao Tong University Joint Institute, Shanghai Jiao Tong University（上海交通大学密西根学院）；Michigan State University（密歇根州立大学）
Achieving high-quality reconstructions from low-dose computed tomography (LDCT) measurements is of much importance in clinical settings. Model-based image reconstruction methods have been proven to be effective in removing artifacts in LDCT. In this work, we propose an approach to learn a rich two-layer clustering-based sparsifying transform model (MCST2), where image patches and their subsequent feature maps (filter residuals) are clustered into groups with different learned sparsifying filters per group. We investigate a penalized weighted least squares (PWLS) approach for LDCT reconstructio…
What Are We Really Measuring? Rethinking Dataset Bias in Web-Scale Natural Image Collections via Unsupervised Semantic Clustering
School of Electrical Engineering, Iran University of Science and Technology (IUST)（伊朗科学技术大学电气工程学院）；Okinawa Institute of Science and Technology (OIST)（冲绳科学技术大学院大学）
Image segmentation in the medical domain is a challenging field owing to poor resolution and limited contrast. The predominantly used conventional segmentation techniques and the thresholding methods suffer from limitations because of heavy dependence on user interactions. Uncertainties prevalent in an image cannot be captured by these techniques. The performance further deteriorates when the images are corrupted by noise, outliers and other artifacts. The objective of this paper is to develop an effective robust fuzzy C- means clustering for segmenting vertebral body from magnetic resonance i…
Monash University（莫纳什大学）；OzGrav: The ARC Centre of Excellence for Gravitational Wave Discovery（OzGrav：ARC引力波发现卓越中心）；University of Warwick（华威大学）；Institute for Globally Distributed Open Research and Education (IGDORE)（全球分布式开放研究与教育研究所（IGDORE））；University of Sheffield（谢菲尔德大学）
Developing an effective automatic classifier to separate genuine sources from artifacts is essential for transient follow-ups in wide-field optical surveys. The identification of transient detections from the subtraction artifacts after the image differencing process is a key step in such classifiers, known as real-bogus classification problem. We apply a self-supervised machine learning model, the deep-embedded self-organizing map (DESOM) to this "real-bogus" classification problem. DESOM combines an autoencoder and a self-organizing map to perform clustering in order to distinguish between r…
A detection-task-specific deep-learning method to improve the quality of sparse-view myocardial perfusion SPECT images
Department of Electrical and Systems Engineering, Washington University, St. Louis, MO, USA（电气与系统工程系，华盛顿大学）；Department of Biomedical Engineering, Washington University, St. Louis, MO, USA（生物医学工程系，华盛顿大学）；Mallinckrodt Institute of Radiology, Washington University, St. Louis, MO, USA（马林克罗德特放射医学研究所，华盛顿大学）
Learning similarity is a key aspect in medical image analysis, particularly in recommendation systems or in uncovering the interpretation of anatomical data in images. Most existing methods learn such similarities in the embedding space over image sets using a single metric learner. Images, however, have a variety of object attributes such as color, shape, or artifacts. Encoding such attributes using a single metric learner is inadequate and may fail to generalize. Instead, multiple learners could focus on separate aspects of these attributes in subspaces of an overarching embedding. This, how…
