SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation
Inception Institute of Artificial Intelligence（起源人工智能研究院）；Tianjin University（天津大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Jiale Cao, Rao Muhammad Anwer, Hisham Cholakkal, Fahad Shahbaz Khan, Yanwei Pang, Ling Shao
Single-stage instance segmentation approaches have recently gained popularity due to their speed and simplicity, but are still lagging behind in accuracy, compared to two-stage methods. We propose a fast single-stage instance segmentation method, called SipMask, that preserves instance-specific spatial information by separating mask prediction of an instance to different sub-regions of a detected bounding-box. Our main contribution is a novel light-weight spatial preservation (SP) module that generates a separate set of spatial coefficients for each sub-region within a bounding-box, leading to improved mask predictions. It also enables accurate delineation of spatially adjacent instances. Further, we introduce a mask alignment weighting loss and a feature alignment scheme to better correlate mask prediction with object detection. On COCO test-dev, our SipMask outperforms the existing single-stage methods. Compared to the state-of-the-art single-stage TensorMask, SipMask obtains an absolute gain of 1.0% (mask AP), while providing a four-fold speedup. In terms of real-time capabilities, SipMask outperforms YOLACT with an absolute gain of 3.0% (mask AP) under similar settings, while operating at comparable speed on a Titan Xp. We also evaluate our SipMask for real-time video instance segmentation, achieving promising results on YouTube-VIS dataset. The source code is available at https://github.com/JialeCao001/SipMask.
The problem of automated car damage assessment presents a major challenge in the auto repair and damage assessment industry. The domain has several application areas ranging from car assessment companies such as car rentals and body shops to accidental damage assessment for car insurance companies. In vehicle assessment, the damage can take any form including scratches, minor and major dents to missing parts. More often, the assessment area has a significant level of noise such as dirt, grease, oil or rush that makes an accurate identification challenging. Moreover, the identification of a par…
Alibaba Group（阿里巴巴集团）；University of Copenhagen（哥本哈根大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Oxford（牛津大学）
Can our video understanding systems perceive objects when a heavy occlusion exists in a scene? To answer this question, we collect a large-scale dataset called OVIS for occluded video instance segmentation, that is, to simultaneously detect, segment, and track instances in occluded scenes. OVIS consists of 296k high-quality instance masks from 25 semantic categories, where object occlusions usually occur. While our human vision systems can understand those occluded instances by contextual reasoning and association, our experiments suggest that current video understanding systems cannot. On the…
Farsee2 Technology（Farsee2科技）；Wuhan University（武汉大学）；Huazhong University of Science and Technology（华中科技大学）；University of South Carolina（南卡罗来纳大学）
Binary image segmentation plays an important role in computer vision and has been widely used in many applications such as image and video editing, object extraction, and photo composition. In this paper, we propose a novel interactive binary image segmentation method based on the Markov Random Field (MRF) framework and the fast bilateral solver (FBS) technique. Specifically, we employ the geodesic distance component to build the unary term. To ensure both computation efficiency and effective responsiveness for interactive segmentation, superpixels are used in computing geodesic distances inst…
Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；Shanghai University（上海大学）；School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Stony Brook University（石溪大学）
As a natural way for human-computer interaction, fixation provides a promising solution for interactive image segmentation. In this paper, we focus on Personal Fixations-based Object Segmentation (PFOS) to address issues in previous studies, such as the lack of appropriate dataset and the ambiguity in fixations-based interaction. In particular, we first construct a new PFOS dataset by carefully collecting pixel-level binary annotation data over an existing fixation prediction dataset, such dataset is expected to greatly facilitate the study along the line. Then, considering characteristics of …
Beijing Normal University（北京师范大学）；Hong Kong Baptist University（香港浸会大学）
We propose a geometric convexity shape prior preservation method for variational level set based image segmentation methods. Our method is built upon the fact that the level set of a convex signed distanced function must be convex. This property enables us to transfer a complicated geometrical convexity prior into a simple inequality constraint on the function. An active set based Gauss-Seidel iteration is used to handle this constrained minimization problem to get an efficient algorithm. We apply our method to region and edge based level set segmentation models including Chan-Vese (CV) model …
Convolutional neural networks for semantic segmentation suffer from low performance at object boundaries. In medical imaging, accurate representation of tissue surfaces and volumes is important for tracking of disease biomarkers such as tissue morphology and shape features. In this work, we propose a novel distance map derived loss penalty term for semantic segmentation. We propose to use distance maps, derived from ground truth masks, to create a penalty term, guiding the network's focus towards hard-to-segment boundary regions. We investigate the effects of this penalizing factor against cro…
FocusSDF: Boundary-Aware Learning for Medical Image Segmentation via Signed Distance Supervision
College of Innovation and Technology (CIT), University of Michigan-Flint（创新与技术学院（CIT），密歇根大学弗林特分校）；College of Information Technology (CIT), United Arab Emirates University（信息科技学院（CIT），阿联酋大学）；Department of Computer Science and Engineering, Oakland University（计算机科学与工程系，奥克兰大学）；Executive Vice-Chair at Department of Neurosurgery, Henry Ford Health System（神经外科部门执行副主席，亨利福特健康系统）
East China Normal University（华东师范大学）；University of British Columbia（不列颠哥伦比亚大学）；Tencent（腾讯）；Tsinghua University（清华大学）
U-Nets have achieved tremendous success in medical image segmentation. Nevertheless, it may suffer limitations in global (long-range) contextual interactions and edge-detail preservation. In contrast, Transformer has an excellent ability to capture long-range dependencies by leveraging the self-attention mechanism into the encoder. Although Transformer was born to model the long-range dependency on the extracted feature maps, it still suffers from extreme computational and spatial complexities in processing high-resolution 3D feature maps. This motivates us to design the efficiently Transforme…
MacFormer: Semantic Segmentation with Fine Object Boundaries
University of Technology Sydney（悉尼科技大学）；Nanjing University（南京大学）；University of Surrey（萨里大学）；Surrey Institute of People-Centred AI（萨里以人为本人工智能研究院）；Institute of Advanced Technology, Nanjing University of Posts and Telecommunications（南京邮电大学先进技术研究院）
GeloVec: Higher Dimensional Geometric Smoothing for Coherent Visual Feature Extraction in Image Segmentation
Department of Computer Science and Engineering（计算机科学与工程系）；Hong Kong University of Science and Technology（香港科技大学）；Division of Integrative Systems and Design（整合系统与设计 division）
University of Twente（特文特大学）；Wuhan University（武汉大学）；Ohio State University（俄亥俄州立大学）
Semantic segmentation has been one of the leading research interests in computer vision recently. It serves as a perception foundation for many fields, such as robotics and autonomous driving. The fast development of semantic segmentation attributes enormously to the large scale datasets, especially for the deep learning related methods. There already exist several semantic segmentation datasets for comparison among semantic segmentation methods in complex urban scenes, such as the Cityscapes and CamVid datasets, where the side views of the objects are captured with a camera mounted on the dri…
BiSe-Unet: A Lightweight Dual-path U-Net with Attention-refined Context for Real-time Medical Image Segmentation
Electrical and Computer Engineering, University of Central Florida（佛罗里达中央大学电气与计算机工程系）；Department of Medicine, University of Central Florida College of Medicine（佛罗里达中央大学医学院医学系）
The information provided by historical documents has always been indispensable in the transmission of human civilization, but it has also made these books susceptible to damage due to various factors. Thanks to recent technology, the automatic digitization of these documents are one of the quickest and most effective means of preservation. The main steps of automatic text digitization can be divided into two stages, mainly: character segmentation and character recognition, where the recognition results depend largely on the accuracy of segmentation. Therefore, in this study, we will only focus…
Region-Aware CAM: High-Resolution Weakly-Supervised Defect Segmentation via Salient Region Perception
School of Instrumentation Science and Engineering, Harbin Institute of Technology（仪器科学与工程学院，哈尔滨工业大学）；Harbin Institute of Technology Suzhou Research Institute（哈尔滨工业大学苏州研究所）
Test-time generative augmentation for medical image segmentation
organization= School of Computer Science；Engineering, Nanjing University of Science；organization= Bioengineering Department；Imperial-X, Imperial College London , city= London , postcode= W12 7SL , country= UK；organization= Digital Medical Research Center, School of Basic Medical Sciences, Fudan University , city= Shanghai , country= China；organization= Shanghai Key Laboratory of MICCAI , city= Shanghai , country= China；organization= School of Biomedical Engineering, Shenzhen University , city= Shenzhen , country= China；organization= Department of Computer Science；Engineering, The Hong Kong University of Science；Engineering, The Chinese University of Hong Kong , city= Hong Kong , country= China；Lung Institute, Imperial College London , city= London , postcode= SW7 2AZ , country= UK；organization= Cardiovascular Research Centre, Royal Brompton Hospital , city= London , postcode= SW3 6NP , country= UK；organization= School of Biomedical Engineering \& Imaging Sciences, King's College London , city= London , postcode= WC2R 2LS , country= UK
An annotated instance segmentation XXL-CT data-set from a historic airplane
Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Deutsches Museum（德国博物馆）
Sparse cubical complexes for efficient topology-preservation in image data
Weill Cornell Medicine（威尔康奈尔医学院）；Technical University of Munich（慕尼黑工业大学）；Imperial College London（伦敦帝国理工学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Cornell Tech（康奈尔科技校区）；Munich Data Science Institute（慕尼黑数据科学研究所）
Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM)（慕尼黑工业大学医疗保健与医学人工智能教席）；TUM University Hospital（慕尼黑工业大学医院）；School of Computation, Information and Technology, TUM（慕尼黑工业大学计算、信息与技术学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Department of Computing, Imperial College London（伦敦帝国理工学院计算机系）；Weill Cornell Medicine, Cornell University（康奈尔大学威尔医学院）
Rethinking Post-Hoc Calibration in Semantic Segmentation
ICube Laboratory, CNRS UMR 7357, University of Strasbourg（斯特拉斯堡大学ICube实验室，CNRS UMR 7357）；CLCC Institut Strauss（斯特劳斯研究所）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Faculty of Mathematics and Computer Science, University of Heidelberg（海德堡大学数学与计算机科学学院）；Medical Faculty Heidelberg, Heidelberg University（海德堡大学医学院）；Pattern Analysis and Learning Group, Dept. of Radiation Oncology, Heidelberg University Hospital（海德堡大学医院放射肿瘤科模式分析与学习组）
Technical University of Darmstadt（达姆施塔特工业大学）；MedCom GmbH（MedCom股份有限公司）
Continual learning protocols are attracting increasing attention from the medical imaging community. In continual environments, datasets acquired under different conditions arrive sequentially; and each is only available for a limited period of time. Given the inherent privacy risks associated with medical data, this setup reflects the reality of deployment for deep learning diagnostic radiology systems. Many techniques exist to learn continuously for image classification, and several have been adapted to semantic segmentation. Yet most struggle to accumulate knowledge in a meaningful manner. …
School of Computer Science and Technology, Zhejiang Normal University（浙江师范大学计算机科学与技术学院）；Metaverse and Artificial Intelligence Institute of Wenzhou University（温州大学元宇宙与人工智能研究院）
Multi-organ segmentation is a critical task in computer-aided diagnosis. While recent deep learning methods have achieved remarkable success in image segmentation, huge variations in organ size and shape challenge their effectiveness in multi-organ segmentation. To address these challenges, we propose a Spatial Prior-Guided Cross Dual Encoder Network (SPG-CDENet), a novel two-stage segmentation paradigm designed to improve multi-organ segmentation accuracy. Our SPG-CDENet consists of two key components: a spatial prior network and a cross dual encoder network. The prior network generates coars…
Computer Vision and Pattern Recognition Unit, Indian Statistical Institute（印度统计研究所计算机视觉与模式识别部）；Østfold University College（东福尔大学学院）；Oslo Metropolitan University（奥斯陆城市大学）
Methods based on convolutional neural networks have improved the performance of biomedical image segmentation. However, most of these methods cannot efficiently segment objects of variable sizes and train on small and biased datasets, which are common for biomedical use cases. While methods exist that incorporate multi-scale fusion approaches to address the challenges arising with variable sizes, they usually use complex models that are more suitable for general semantic segmentation problems. In this paper, we propose a novel architecture called Multi-Scale Residual Fusion Network (MSRF-Net),…
Boosting Semi-Supervised Medical Image Segmentation via Masked Image Consistency and Discrepancy Learning
School of Information and Communication Engineering, Beijing University of Posts and Telecommunications（北京邮电大学信息与通信工程学院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）
GLCP: Global-to-Local Connectivity Preservation for Tubular Structure Segmentation
Department of Eye and Vision Sciences, University of Liverpool, Liverpool, UK（眼科学与视觉科学系，利物浦大学，英国利物浦）；Department of Computer Science, University of Exeter, Exeter, UK（计算机科学系，埃克塞特大学，英国埃克塞特）；Ningbo Institute of Materials Technology and Engineering, CAS, Ningbo, China（宁波材料技术与工程研究所，中国科学院，中国宁波）
Department of Electrical Engineering, Indian Institute of Technology Ropar, India（电气工程系，印度理工学院罗帕尔学院）；Department of Computer Science and Engineering, Indian Institute of Technology Ropar, India（计算机科学与工程系，印度理工学院罗帕尔学院）；Artificial Intelligence and Data Engineering, Indian Institute of Technology Ropar, India（人工智能与数据工程，印度理工学院罗帕尔学院）
