Key Laboratory of Intelligent Information Processing, ICT, CAS(中国科学院计算技术研究所智能信息处理重点实验室);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院);University of Chinese Academy of Sciences(中国科学院大学)
We present a unified, promptable model capable of simultaneously segmenting, recognizing, and captioning anything. Unlike SAM, we aim to build a versatile region representation in the wild via visual prompting. To achieve this, we train a generalizable model with massive segmentation masks, \eg, SA-1B masks, and semantic priors from a pre-trained CLIP model with 5 billion parameters. Specifically, we construct a promptable image decoder by adding a semantic token to each mask token. The semantic token is responsible for learning the semantic priors in a predefined concept space. Through joint optimization of segmentation on mask tokens and concept prediction on semantic tokens, our model exhibits strong regional recognition and localization capabilities. For example, an additional 38M-parameter causal text decoder trained from scratch sets a new record with a CIDEr score of 164.7 on the Visual Genome region captioning task. We believe this model can be a versatile region-level image tokenizer, capable of encoding general-purpose region context for a broad range of visual perception tasks. Code and models are available at {\footnotesize \url{https://github.com/baaivision/tokenize-anything}}.
Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification
Artificial Intelligence Innovation and Incubation Institute, Fudan University(复旦大学人工智能创新与孵化研究院);Shanghai Academy of Artificial Intelligence for Science(上海科学人工智能研究院)
Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation
The Chinese University of Hong Kong(香港中文大学);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Institute of Medical Intelligence and XR, The Chinese University of Hong Kong(香港中文大学医学智能与XR研究所)
提出 CPC-SAM,在双分支框架中通过 SAM 交叉提示和提示一致性正则化利用标注与未标注数据,显著提升半监督医学图像分割性能。
Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation
National Engineering Research Center for Multimedia Software(国家多媒体软件工程技术研究中心);Medical Artificial Intelligence Research Institute of Renmin Hospital, Wuhan University(武汉大学人民医院医学人工智能研究院);School of Computer Science, Central China Normal University(华中师范大学计算机学院)
Any to Full: Prompting Depth Anything for Depth Completion in One Stage
Rutgers University(罗格斯大学);Michigan State University(密歇根州立大学);JD Logistics(京东物流);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France(斯特拉斯堡大学、法国国家科学研究中心、法国国家医学研究院、ICube、UMR7357、法国斯特拉斯堡);Department of General Surgery, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan(高雄长庚纪念医院外科部、长庚大学医学院)
ViPSAM: Visual Prompting Medical Image Segmentation Using Segment Anything Model
Department of Artificial Intelligence, Sungkyunkwan University, Republic of Korea(人工智能系,成均馆大学,大韩民国);Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Republic of Korea(放射肿瘤科,三星医疗中心,成均馆大学医学院,大韩民国);Department of MetaBioHealth, Sungkyunkwan University, Republic of Korea(元生物健康系,成均馆大学,大韩民国)
Technical University of Darmstadt(达姆施塔特工业大学);German Research Center for AI (DFKI)(德国人工智能研究中心);Hessian Center for AI (hessian.AI)(黑森人工智能中心);Eindhoven University of Technology(埃因霍温理工大学);Center for European Research in Trusted Artificial Intelligence (CERTAIN)(欧洲可信人工智能研究中心)
Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)
Lund University Humanities Lab \& Dept. of Psychology, Lund University Lund Sweden;Lund University Humanities Lab Lund Sweden;Lund University Humanities Lab \& Dept. of Psychology, Lund University;Lund University Humanities Lab
CUHK(香港中文大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Peking University(北京大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Prompting DirectSAM for Semantic Contour Extraction in Remote Sensing Images
College of Computer Science and Software Engineering, Hohai University(河海大学计算机与软件学院);Hong Kong University of Science and Technology(香港科技大学);School of Computer and Electronic Information, Nanjing Normal University(南京师范大学计算机与电子信息学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
SAM2-SGP: Enhancing SAM2 for Medical Image Segmentation via Support-Set Guided Prompting
J. Crayton Pruitt Family Department of Biomedical Engineering University of Florida;Department of Electrical \& Computer Engineering University of Florida
AuralSAM2: Enabling SAM2 Hear Through Pyramid Audio-Visual Feature Prompting
Department of Engineering Science, University of Oxford(牛津大学工程科学系);Australian Institute for Machine Learning, Adelaide University(阿德莱德大学人工智能研究所);Stanford University(斯坦福大学);University of Central Florida(佛罗里达中央大学);University of Surrey(萨里大学)
Unveiling the Potential of Segment Anything Model 2 for RGB-Thermal Semantic Segmentation with Language Guidance
School of Artificial Intelligence and Robotics, Hunan University(人工智能与机器人学院,湖南大学);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(机器人视觉感知与控制技术国家工程研究中心,湖南大学);AI Thrust, Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州)人工智能方向)
Segment Anything Model is a Good Teacher for Local Feature Learning
The University of Hong Kong(香港大学);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Northwestern University(西北大学);Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)(计算能力网络与信息安全重点实验室,教育部长江计算机科学中心(济南国家超算中心),齐鲁工业大学(山东科学院));Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science(山东省计算机网络重点实验室,山东省计算机科学基础研究中心)
RadSAM: Segmenting 3D radiological images with a 2D promptable model
Raidium;Université de Paris Cité(巴黎大学);AP-HP(法国国家卫生与医学研究院);Hôpital Européen Georges Pompidou(欧仁·蓬皮杜医院);Department of Vascular and Oncological Interventional Radiology(血管和肿瘤介入放射科);HEKA INRIA;INSERM PARCC U 970(法国国家医学研究院 PARCC U 970)