The Segment Anything Model (SAM) excels at generating precise object masks from input prompts but lacks semantic awareness, failing to associate its generated masks with specific object categories. To address this limitation, we propose U-SAM, a novel framework that imbibes semantic awareness into SAM, enabling it to generate targeted masks for user-specified object categories. Given only object class names as input from the user, U-SAM provides pixel-level semantic annotations for images without requiring any labeled/unlabeled samples from the test data distribution. Our approach leverages synthetically generated or web crawled images to accumulate semantic information about the desired object classes. We then learn a mapping function between SAM's mask embeddings and object class labels, effectively enhancing SAM with granularity-specific semantic recognition capabilities. As a result, users can obtain meaningful and targeted segmentation masks for specific objects they request, rather than generic and unlabeled masks. We evaluate U-SAM on PASCAL VOC 2012 and MSCOCO-80, achieving significant mIoU improvements of +17.95% and +5.20%, respectively, over state-of-the-art methods. By transforming SAM into a semantically aware segmentation model, U-SAM offers a practical and flexible solution for pixel-level annotation across diverse and unseen domains in a resource-constrained environment.
While GANs have shown success in realistic image generation, the idea of using GANs for other tasks unrelated to synthesis is underexplored. Do GANs learn meaningful structural parts of objects during their attempt to reproduce those objects? In this work, we test this hypothesis and propose a simple and effective approach based on GANs for semantic part segmentation that requires as few as one label example along with an unlabeled dataset. Our key idea is to leverage a trained GAN to extract pixel-wise representation from the input image and use it as feature vectors for a segmentation networ…
Pinscreen（Pinscreen公司）；University of Southern California（南加州大学）
We introduce the concept of unconstrained real-time 3D facial performance capture through explicit semantic segmentation in the RGB input. To ensure robustness, cutting edge supervised learning approaches rely on large training datasets of face images captured in the wild. While impressive tracking quality has been demonstrated for faces that are largely visible, any occlusion due to hair, accessories, or hand-to-face gestures would result in significant visual artifacts and loss of tracking accuracy. The modeling of occlusions has been mostly avoided due to its immense space of appearance var…
University of Birmingham（伯明翰大学）；Zhejiang University（浙江大学）；University of Victoria（维多利亚大学）
We propose a novel optimization framework that crops a given image based on user description and aesthetics. Unlike existing image cropping methods, where one typically trains a deep network to regress to crop parameters or cropping actions, we propose to directly optimize for the cropping parameters by repurposing pre-trained networks on image captioning and aesthetic tasks, without any fine-tuning, thereby avoiding training a separate network. Specifically, we search for the best crop parameters that minimize a combined loss of the initial objectives of these networks. To make the optimizati…
SEG-SAM: Semantic-Guided SAM for Unified Medical Image Segmentation
School of Electronic and Information Engineering, South China University of Technology（华南理工大学电子与信息工程学院）；Pazhou Lab（琶洲实验室）；College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）
SAM3-Adapter: Efficient Adaptation of Segment Anything 3 for Camouflage Object Segmentation, Shadow Detection, and Medical Image Segmentation
KOKONI, Moxin (Huzhou) Tech. Co., LTD（摩西（湖州）科技有限公司）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Information Engineering, Huzhou University（湖州大学信息工程学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（新加坡南洋理工大学电子与电气工程学院）；College of Computing and Data Science, Nanyang Technological University（新加坡南洋理工大学计算与数据科学学院）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学与技术学院）
Revisiting foundation models for cell instance segmentation
Georg-August-University Göttingen, Institute of Computer Science（哥廷根大学计算机科学学院）；CAIMed - Lower Saxony Center for AI & Causal Methods in Medicine, Göttingen（下萨克森人工智能与因果医学方法中心，哥廷根）；Cluster of Excellence Multiscale Bioimaging (MBExC), Georg-August-University Göttingen（多尺度生物成像（MBExC）集群，哥廷根大学）
SAMBA: A Trainable Segmentation Web-App with Smart Labelling
Imperial College London（帝国理工学院）；Department of Materials, Imperial College London（帝国理工学院材料系）；Dyson School of Design Engineering, Imperial College London（帝国理工学院戴森设计工程学院）
SAMBA 是一款基于浏览器和云端的可训练分割工具，结合 Meta 的 SAM 模型与随机森林分类器，实现快速智能标注和稳健可泛化的材料科学图像分割。
Toyota Technological Institute at Chicago（丰田工业大学芝加哥分校）
We propose an approach for learning category-level semantic segmentation purely from image-level classification tags indicating presence of categories. It exploits localization cues that emerge from training classification-tasked convolutional networks, to drive a "self-supervision" process that automatically labels a sparse, diverse training set of points likely to belong to classes of interest. Our approach has almost no hyperparameters, is modular, and allows for very fast training of segmentation in less than 3 minutes. It obtains competitive results on the VOC 2012 segmentation benchmark.…
SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More
KOKONI, Moxin (Huzhou) Tech. Co., LTD（莫西（湖州）科技有限公司KOKONI）；Beihang University（北京航空航天大学）；Zhejiang University（浙江大学）；Huzhou University（湖州师范学院）；Singapore University of Technology and Design（新加坡科技设计大学）
Learnable Prompting SAM-induced Knowledge Distillation for Semi-supervised Medical Image Segmentation
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Institute of High-Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
University of Oxford（牛津大学）；National University of Singapore（新加坡国立大学）；University of Alberta（阿尔伯塔大学）；A*STAR（新加坡科技研究局）；Carnegie Mellon University（卡内基梅隆大学）；Singapore Eye Research Institute（新加坡眼科研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Hongkong（香港大学）
本文提出Medical SAM Adapter（Med-SA），通过轻量适配技术将医学知识融入SAM，利用SD-Trans和HyP-Adpt适配2D模型至3D医学图像，在17个任务上以仅2%参数更新超越SOTA方法。
SRPR-Net: Semantic and Relational Prompt Refinement for Automated SAM-based Instance Segmentation
Ruijin-XJTLU Intelligent Medicine Institute, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（瑞金-西交利物浦智能医学研究所，瑞金医院，上海交通大学医学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）
School of EEE（电子工程系）；Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research（信息通信研究所）；A*STAR；College of Computer Science（计算机科学学院）；Nankai University（南开大学）；State Key Laboratory of Public Big Data（公共大数据国家重点实验室）；Guizhou University（贵州大学）；Fudan Vision and Learning Lab（复旦大学视觉与学习实验室）
WPS-SAM: Towards Weakly-Supervised Part Segmentation with Foundation Models
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model
Wuhan University（武汉大学）；The University of Sydney（悉尼大学）；National Engineering Research Center of Speech and Language Information Processing（国家语音及语言信息处理工程技术研究中心）
The foreground segmentation algorithms suffer performance degradation in the presence of various challenges such as dynamic backgrounds, and various illumination conditions. To handle these challenges, we present a foreground segmentation method, based on generative adversarial network (GAN). We aim to segment foreground objects in the presence of two aforementioned major challenges in background scenes in real environments. To address this problem, our presented GAN model is trained on background image samples with dynamic changes, after that for testing the GAN model has to generate the same…
BoxCell: Leveraging SAM for Cell Segmentation with Box Supervision
India Institute Of Technology, Yardi School of Artificial Intelligence（印度理工学院，人工智能学院）；India Institute Of Technology, Department of Computer Science and Engineering（印度理工学院，计算机科学与工程系）；Indian Institute of Science, Department of Electrical Communication Engineering（印度科学研究院，电子通信工程系）
Barcelona Supercomputing Center（巴塞罗那超级计算中心）；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）
Image segmentation methods are usually trained with pixel-level annotations, which require significant human effort to collect. The most common solution to address this constraint is to implement weakly-supervised pipelines trained with lower forms of supervision, such as bounding boxes or scribbles. Another option are semi-supervised methods, which leverage a large amount of unlabeled data and a limited number of strongly-labeled samples. In this second setup, samples to be strongly-annotated can be selected randomly or with an active learning mechanism that chooses the ones that will maximiz…
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；The Chinese University of Hong Kong（香港中文大学）；Lehigh University（理海大学）；Harvard University（哈佛大学）；The University of Georgia（佐治亚大学）
