DAWA: Dynamic Ambiguity-Wise Adaptation for Real-Time Domain Adaptive Semantic Segmentation
School of Computer Science, Wuhan University（武汉大学计算机学院）；Hangzhou Institute of Technology, Xidian University（西安电子科技大学杭州研究院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Department of Electrical Engineering（电气工程系）；the Institute of Communications Engineering, National Tsing Hua University（清华大学通信工程研究所）
Taorong Liu, Zhen Zhang, Liang Liao, Jing Xiao, Chia-Wen Lin
Test-time domain adaption (TTDA) for semantic segmentation aims to adapt a segmentation model trained on a source domain to a target domain for inference on-the-fly, where both efficiency and effectiveness are critical. However, existing TTDA methods either rely on costly frame-wise optimization or assume unrealistic domain shifts, resulting in poor adaptation efficiency and continuous semantic ambiguities. To address these challenges, we propose a real-time framework for TTDA semantic segmentation, called Dynamic Ambiguity-Wise Adaptation (DAWA), which adaptively detects domain shifts and dynamically adjusts the learning strategies to mitigate continuous ambiguities in the test time. Specifically, we introduce the Dynamic Ambiguous Patch Mask (DAP Mask) strategy, which dynamically identifies and masks highly disturbed regions to prevent error accumulation in ambiguous classes. Furthermore, we present the Dynamic Ambiguous Class Mix (DAC Mix) strategy that leverages vision-language models to group semantically similar classes and augment the target domain with a meta-ambiguous class buffer. Extensive experiments on widely used TTDA benchmarks demonstrate that DAWA consistently outperforms state-of-the-art methods, while maintaining real-time inference speeds of approximately 40 FPS.
Beyond Boundary Noise: Aggregated Aleatoric Uncertainty Fails to Capture Presence Ambiguity in 3D Lung Nodule Segmentation
Fraunhofer Heinrich-Hertz-Institut（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；The Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所（BIFOLD））
National University of Defense Technology（国防科技大学）；Shenzhen University（深圳大学）
Context has proven to be one of the most important factors in object layout reasoning for 3D scene understanding. Existing deep contextual models either learn holistic features for context encoding or rely on pre-defined scene templates for context modeling. We argue that scene understanding benefits from object relation reasoning, which is capable of mitigating the ambiguity of 3D object detections and thus helps locate and classify the 3D objects more accurately and robustly. To achieve this, we propose a novel 3D relation module (3DRM) which reasons about object relations at pair-wise level…
Smoke segmentation is essential to precisely localize wildfire so that it can be extinguished in an early phase. Although deep neural networks have achieved promising results on image segmentation tasks, they are prone to be overconfident for smoke segmentation due to its non-rigid shape and transparent appearance. This is caused by both knowledge level uncertainty due to limited training data for accurate smoke segmentation and labeling level uncertainty representing the difficulty in labeling ground-truth. To effectively model the two types of uncertainty, we introduce a Bayesian generative …
University of Chinese Academy of Sciences（中国科学院大学）
Few-shot segmentation is challenging because objects within the support and query images could significantly differ in appearance and pose. Using a single prototype acquired directly from the support image to segment the query image causes semantic ambiguity. In this paper, we propose prototype mixture models (PMMs), which correlate diverse image regions with multiple prototypes to enforce the prototype-based semantic representation. Estimated by an Expectation-Maximization algorithm, PMMs incorporate rich channel-wised and spatial semantics from limited support images. Utilized as representat…
Northeastern University（东北大学）；Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University（东北大学医学影像智能计算教育部重点实验室）；Amii, University of Alberta（阿尔伯塔大学阿尔伯塔机器智能研究所）
Most recent semantic segmentation methods adopt a U-Net framework with an encoder-decoder architecture. It is still challenging for U-Net with a simple skip connection scheme to model the global multi-scale context: 1) Not each skip connection setting is effective due to the issue of incompatible feature sets of encoder and decoder stage, even some skip connection negatively influence the segmentation performance; 2) The original U-Net is worse than the one without any skip connection on some datasets. Based on our findings, we propose a new segmentation framework, named UCTransNet (with a pro…
Unsupervised semantic segmentation aims to discover groupings within and across images that capture object and view-invariance of a category without external supervision. Grouping naturally has levels of granularity, creating ambiguity in unsupervised segmentation. Existing methods avoid this ambiguity and treat it as a factor outside modeling, whereas we embrace it and desire hierarchical grouping consistency for unsupervised segmentation. We approach unsupervised segmentation as a pixel-wise feature learning problem. Our idea is that a good representation shall reveal not just a particular l…
Multi-object tracking remains challenging due to frequent occurrence of occlusions and outliers. In order to handle this problem, we propose an Approximation-Shrink Scheme for sequential optimization. This scheme is realized by introducing an Ambiguity-Clearness Graph to avoid conflicts and maintain sequence independent, as well as a sliding window optimization framework to constrain the size of state space and guarantee convergence. Based on this window-wise framework, the states of targets are clustered in a self-organizing manner. Moreover, we show that the traditional online and batch trac…
KGEdit: Ambiguity-Aware Knowledge Graphs for Training-Free Precise Video Generation and Editing
Waseda University, Japan（日本早稻田大学）；College of Computer Engineering, Jimei University（集美大学计算机工程学院）；Department of Language Science and Technology, The Hong Kong Polytechnic University（香港理工大学语言科学与技术系）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Understanding Image2Video Domain Shift in Food Segmentation: An Instance-level Analysis on Apples
Interdisciplinary Program in Artificial Intelligence, Seoul National University（人工智能交叉学科项目，首尔国立大学）；Department of Biological Environmental Science, College of Life Science and Biotechnology, Dongguk University（生命科学与生物技术学院生物环境科学系，东国大学）；Department of Food Science and Biotechnology, Dongguk University（食品科学与生物技术系，东国大学）
OMUDA: Omni-level Masking for Unsupervised Domain Adaptation in Semantic Segmentation
School of Mechanical Engineering, Chengdu University（成都大学机械工程学院）；School of Information Science and Technology, Harbin Institute of Technology（哈尔滨工业大学信息科学与技术学院）；School of Computer Science and Informatics, Cardiff University（卡的夫大学计算机科学与信息学院）；School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机科学学院）；College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学与技术学院）
Zhejiang University（浙江大学）；Singapore University of Technology and Design（新加坡科技设计大学）；University of Tübingen（图宾根大学）；MPI-IS（马克斯·普朗克智能系统研究所）
Large-scale training data with high-quality annotations is critical for training semantic and instance segmentation models. Unfortunately, pixel-wise annotation is labor-intensive and costly, raising the demand for more efficient labeling strategies. In this work, we present a novel 3D-to-2D label transfer method, Panoptic NeRF, which aims for obtaining per-pixel 2D semantic and instance labels from easy-to-obtain coarse 3D bounding primitives. Our method utilizes NeRF as a differentiable tool to unify coarse 3D annotations and 2D semantic cues transferred from existing datasets. We demonstrat…
DGSeg: Dynamic Gating of Semantic-Spatial Guided Predictions for Reasoning Segmentation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Nanjing Artificial Intelligence Research of IA（中国科学院自动化所南京人工智能研究院）
