We present a new framework for semantic segmentation without annotations via clustering. Off-the-shelf clustering methods are limited to curated, single-label, and object-centric images yet real-world data are dominantly uncurated, multi-label, and scene-centric. We extend clustering from images to pixels and assign separate cluster membership to different instances within each image. However, solely relying on pixel-wise feature similarity fails to learn high-level semantic concepts and overfits to low-level visual cues. We propose a method to incorporate geometric consistency as an inductive bias to learn invariance and equivariance for photometric and geometric variations. With our novel learning objective, our framework can learn high-level semantic concepts. Our method, PiCIE (Pixel-level feature Clustering using Invariance and Equivariance), is the first method capable of segmenting both things and stuff categories without any hyperparameter tuning or task-specific pre-processing. Our method largely outperforms existing baselines on COCO and Cityscapes with +17.5 Acc. and +4.5 mIoU. We show that PiCIE gives a better initialization for standard supervised training. The code is available at https://github.com/janghyuncho/PiCIE.
CARL: A Framework for Equivariant Image Registration
UNC Chapel Hill(北卡罗来纳大学教堂山分校);Université Gustave Eiffel(Gustave Eiffel 大学);INRIA Paris(法国国家信息与自动化研究所巴黎分部);University of Salzburg(萨尔茨堡大学);Harvard Medical School(哈佛医学院);UC San Diego(加州大学圣迭戈分校)
DULDA: Dual-domain Unsupervised Learned Descent Algorithm for PET image reconstruction
Zhejiang University(浙江大学);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);University of Florida(佛罗里达大学);Massachusetts General Brigham(麻省总医院布莱根医疗系统)
SegMatch: A semi-supervised learning method for surgical instrument segmentation
School of Biomedical Engineering & Imaging Sciences, King’s College London(生物医学工程与影像科学学院,伦敦国王学院);Harvard Medical School, Harvard University(哈佛医学院,哈佛大学);College of Electronic and Information Engineering, Tongji University(同济大学电子与信息工程学院)