Removing supervision in semantic segmentation is still tricky. Current approaches can deal with common categorical patterns yet resort to multi-stage architectures. We design a novel end-to-end model leveraging local-global patch matching to predict categories, good localization, area and shape of objects for semantic segmentation. The local-global matching is, in turn, compelled by optimal transport plans fulfilling area constraints nearing a solution for exact shape prediction. Our model attains state-of-the-art in Weakly Supervised Semantic Segmentation, only image-level labels, with 75% mIoU on PascalVOC2012 val set and 46% on MS-COCO2014 val set. Dropping the image-level labels and clustering self-supervised learned features to yield pseudo-multi-level labels, we obtain an unsupervised model for semantic segmentation. We also attain state-of-the-art on Unsupervised Semantic Segmentation with 43.6% mIoU on PascalVOC2012 val set and 19.4% on MS-COCO2014 val set. The code is available at https://github.com/deepplants/PC2M.
Debiased Collaborative Filtering with Kernel-Based Causal Balancing
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Technology and Business University（北京工商大学）；Renmin University of China（中国人民大学）；Tsinghua University（清华大学）
The panoptic segmentation task requires a unified result from semantic and instance segmentation outputs that may contain overlaps. However, current studies widely ignore modeling overlaps. In this study, we aim to model overlap relations among instances and resolve them for panoptic segmentation. Inspired by scene graph representation, we formulate the overlapping problem as a simplified case, named scene overlap graph. We leverage each object's category, geometry and appearance features to perform relational embedding, and output a relation matrix that encodes overlap relations. In order to …
Open-Vocabulary Semantic Segmentation with Image Embedding Balancing
National Key Laboratory of Multispectral Information Intelligent Processing Technology（多光谱信息智能处理技术国家重点实验室）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
本文提出 EBSeg 框架，通过 AdaB Decoder 自适应平衡训练类与新类图像嵌入，并结合 SSC Loss 与冻结 SAM 编码器，提升开放词汇语义分割的泛化和空间定位性能。
Decoupled Pseudo-labeling for Semi-Supervised Monocular 3D Object Detection
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Baidu Inc.（百度公司）；GuangDong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）
National Taiwan University（台湾大学）；Chang Gung Memorial Hospital（长庚纪念医院）；Chang Gung University（长庚大学）
Automated vascular segmentation on optical coherence tomography angiography (OCTA) is important for the quantitative analyses of retinal microvasculature in neuroretinal and systemic diseases. Despite recent improvements, artifacts continue to pose challenges in segmentation. Our study focused on removing the speckle noise artifact from OCTA images when performing segmentation. Speckle noise is common in OCTA and is particularly prominent over large non-perfusion areas. It may interfere with the proper assessment of retinal vasculature. In this study, we proposed a novel Supervision Vessel Seg…
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Saarland Informatics Campus（萨尔兰信息学园区）；Stanford University（斯坦福大学）
Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture approaches either required expensive multi-view setups or did not recover dense space-time coherent geometry with frame-to-frame correspondences. We propose a novel deep learning approach for monocular dense human performance capture. Our method is trained in a weakly supervised manner based on multi-view supervision completely removing the need for training data with 3D ground truth annotations. The network architec…
Centrum Wiskunde & Informatica（数学与计算机科学研究所）；Leiden University（莱顿大学）；Eindhoven University of Technology（埃因霍温理工大学）
We present three parallel-beam tomographic datasets of 94 apples with internal defects along with defect label files. The datasets are prepared for development and testing of data-driven, learning-based image reconstruction, segmentation and post-processing methods. The three versions are a noiseless simulation; simulation with added Gaussian noise, and with scattering noise. The datasets are based on real 3D X-ray CT data and their subsequent volume reconstructions. The ground truth images, based on the volume reconstructions, are also available through this project. Apples contain various de…
Nara Institute of Science and Technology（奈良先端科学技术大学院大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；OMRON SINIC X Corp.（欧姆龙SINIC X公司）
Unsupervised image captioning is a challenging task that aims at generating captions without the supervision of image-sentence pairs, but only with images and sentences drawn from different sources and object labels detected from the images. In previous work, pseudo-captions, i.e., sentences that contain the detected object labels, were assigned to a given image. The focus of the previous work was on the alignment of input images and pseudo-captions at the sentence level. However, pseudo-captions contain many words that are irrelevant to a given image. In this work, we investigate the effect o…
Fairness Without Labels: Pseudo-Balancing for Bias Mitigation in Face Gender Classification
National Institute of Informatics, Japan（日本信息机构国家研究所）；INESC-ID, Instituto Superior Técnico, University of Lisbon, Portugal（里斯本大学技术高等学院、INESC-ID，葡萄牙）；University of Amsterdam, The Netherlands（荷兰阿姆斯特丹大学）；LIMSI, CNRS / Université Paris-Saclay, France（法国CNRS/巴黎萨克雷大学LIMSI）
École Polytechnique（巴黎综合理工学院）；CNRS（法国国家科学研究中心）；IP Paris（巴黎理工学院）；AMIAD；École Nationale des Ponts et Chaussées（巴黎路桥学院）；UGE（巴黎第八大学）；UC Berkeley（加州大学伯克利分校）
JointViT: Modeling Oxygen Saturation Levels with Joint Supervision on Long-Tailed OCTA
Flinders University（弗林德斯大学）；The Australian National University（澳大利亚国立大学）；The University of Adelaide（阿德莱德大学）；Guangdong Technion - Israel Institute of Technology（广东以色列理工学院）；The University of Sydney（悉尼大学）
University of Colorado at Colorado Springs（科罗拉多大学科罗拉多斯普林斯分校）
Attribute recognition, particularly facial, extracts many labels for each image. While some multi-task vision problems can be decomposed into separate tasks and stages, e.g., training independent models for each task, for a growing set of problems joint optimization across all tasks has been shown to improve performance. We show that for deep convolutional neural network (DCNN) facial attribute extraction, multi-task optimization is better. Unfortunately, it can be difficult to apply joint optimization to DCNNs when training data is imbalanced, and re-balancing multi-label data directly is str…
