Fingerprint-based identification systems achieve higher accuracy when a slap containing multiple fingerprints of a subject is used instead of a single fingerprint. However, segmenting or auto-localizing all fingerprints in a slap image is a challenging task due to the different orientations of fingerprints, noisy backgrounds, and the smaller size of fingertip components. The presence of slap images in a real-world dataset where one or more fingerprints are rotated makes it challenging for a biometric recognition system to localize and label the fingerprints automatically. Improper fingerprint localization and finger labeling errors lead to poor matching performance. In this paper, we introduce a method to generate arbitrary angled bounding boxes using a deep learning-based algorithm that precisely localizes and labels fingerprints from both axis-aligned and over-rotated slap images. We built a fingerprint segmentation model named CRFSEG (Clarkson Rotated Fingerprint segmentation Model) by updating the previously proposed CFSEG model which was based on traditional Faster R-CNN architecture [21]. CRFSEG improves upon the Faster R-CNN algorithm with arbitrarily angled bounding boxes that allow the CRFSEG to perform better in challenging slap images. After training the CRFSEG algorithm on a new dataset containing slap images collected from both adult and children subjects, our results suggest that the CRFSEG model was invariant across different age groups and can handle over-rotated slap images successfully. In the Combined dataset containing both normal and rotated images of adult and children subjects, we achieved a matching accuracy of 97.17%, which outperformed state-of-the-art VeriFinger (94.25%) and NFSEG segmentation systems (80.58%).
A fingerprint region of interest (roi) segmentation algorithm is designed to separate the foreground fingerprint from the background noise. All the learning based state-of-the-art fingerprint roi segmentation algorithms proposed in the literature are benchmarked on scenarios when both training and testing databases consist of fingerprint images acquired from the same sensors. However, when testing is conducted on a different sensor, the segmentation performance obtained is often unsatisfactory. As a result, every time a new fingerprint sensor is used for testing, the fingerprint roi segmentati…
Michigan State University（密歇根州立大学）；Dayalbagh Educational Institute（达亚尔巴格教育学院）；Saran Ashram Hospital（萨兰阿什拉姆医院）
With a number of emerging applications requiring biometric recognition of children (e.g., tracking child vaccination schedules, identifying missing children and preventing newborn baby swaps in hospitals), investigating the temporal stability of biometric recognition accuracy for children is important. The persistence of recognition accuracy of three of the most commonly used biometric traits (fingerprints, face and iris) has been investigated for adults. However, persistence of biometric recognition accuracy has not been studied systematically for children in the age group of 0-4 years. Given…
Verifying an identity claim by fingerprint recognition is a commonplace experience for millions of people in their daily life, e.g. for unlocking a tablet computer or smartphone. The first processing step after fingerprint image acquisition is segmentation, i.e. dividing a fingerprint image into a foreground region which contains the relevant features for the comparison algorithm, and a background region. We propose a novel segmentation method by global three-part decomposition (G3PD). Based on global variational analysis, the G3PD method decomposes a fingerprint image into cartoon, texture an…
We present a simple but effective method for automatic latent fingerprint segmentation, called SegFinNet. SegFinNet takes a latent image as an input and outputs a binary mask highlighting the friction ridge pattern. Our algorithm combines fully convolutional neural network and detection-based approaches to process the entire input latent image in one shot instead of using latent patches. Experimental results on three different latent databases (i.e. NIST SD27, WVU, and an operational forensic database) show that SegFinNet outperforms both human markup for latents and the state-of-the-art laten…
Many fingerprint recognition systems capture four fingerprints in one image. In such systems, the fingerprint processing pipeline must first segment each four-fingerprint slap into individual fingerprints. Note that most of the current fingerprint segmentation algorithms have been designed and evaluated using only adult fingerprint datasets. In this work, we have developed a human-annotated in-house dataset of 15790 slaps of which 9084 are adult samples and 6706 are samples drawn from children from ages 4 to 12. Subsequently, the dataset is used to evaluate the matching performance of the NFSE…
Institute for Mathematical Stochastics, University of Göttingen（哥廷根大学数学随机过程研究所）；Felix-Bernstein-Institute for Mathematical Statistics in the Biosciences, University of Göttingen（哥廷根大学生物科学数学统计费利克斯·伯恩斯坦研究所）
Fingerprint recognition plays an important role in many commercial applications and is used by millions of people every day, e.g. for unlocking mobile phones. Fingerprint image segmentation is typically the first processing step of most fingerprint algorithms and it divides an image into foreground, the region of interest, and background. Two types of error can occur during this step which both have a negative impact on the recognition performance: 'true' foreground can be labeled as background and features like minutiae can be lost, or conversely 'true' background can be misclassified as fore…
Recursive Class Connectivity Classification (R3C) Applied to Binary Image Segmentation for Improved Infant Fingerprint Enhancement
Infant.ID Ltda（Infant.ID公司）；Graduate Program in Production and Systems Engineering (PPGEPS), Federal University of Technology-Paran (UTFPR)（生产与系统工程硕士项目，联邦技术大学-巴拉那（UTFPR））
Latent fingerprint has the practical value to identify the suspects who have unintentionally left a trace of fingerprint in the crime scenes. However, designing a fully automated latent fingerprint matcher is a very challenging task as it needs to address many challenging issues including the separation of overlapping structured patterns over the partial and poor quality latent fingerprint image, and finding a match against a large background database that would have different resolutions. Currently there is no fully automated latent fingerprint matcher available to the public and most literat…
Birla Institute of Technology and Science Pilani – KK Birla Goa Campus（贝拉理工学院皮拉尼分校–KK·贝拉果阿校区）；Indian Institute of Science Education and Research, Pune（印度科学教育与研究学院浦那分校）；Vellore Institute of Technology, Bhopal（韦洛尔理工学院博帕尔分校）
Biometric Authentication like Fingerprints has become an integral part of the modern technology for authentication and verification of users. It is pervasive in more ways than most of us are aware of. However, these fingerprint images deteriorate in quality if the fingers are dirty, wet, injured or when sensors malfunction. Therefore, extricating the original fingerprint by removing the noise and inpainting it to restructure the image is crucial for its authentication. Hence, this paper proposes a deep learning approach to address these issues using Generative (GAN) and Segmentation models. Qu…
Nowadays research has expanded to extracting auxiliary information from various biometric techniques like fingerprints, face, iris, palm and voice . This information contains some major features like gender, age, beard, mustache, scars, height, hair, skin color, glasses, weight, facial marks and tattoos. All this information contributes strongly to identification of human. The major challenges that come across face recognition are to find age & gender of the person. This paper contributes a survey of various face recognition techniques for finding the age and gender. The existing techniques ar…
Molecular and Neuroimaging Engineering Research Center of MOE, School of Life Science and Technology, Xidian University（教育部分子与神经影像工程研究中心，西安电子科技大学生命科学技术学院）
Singular points detection is one of the most classical and important problem in the field of fingerprint recognition. However, current detection rates of singular points are still unsatisfactory, especially for low-quality fingerprints. Compared with traditional image processing-based detection methods, methods based on deep learning only need the original fingerprint image but not the fingerprint orientation field. In this paper, different from other detection methods based on deep learning, we treat singular points detection as a semantic segmentation problem and just use few data for traini…
Center for Visual Information Technology (CVIT), IIIT-Hyderabad（海得拉巴国际信息技术研究所视觉信息技术中心）
Fingerprint is a common biometric used for authentication and verification of an individual. These images are degraded when fingers are wet, dirty, dry or wounded and due to the failure of the sensors, etc. The extraction of the fingerprint from a degraded image requires denoising and inpainting. We propose to address these problems with an end-to-end trainable Convolutional Neural Network based architecture called FPD-M-net, by posing the fingerprint denoising and inpainting problem as a segmentation (foreground) task. Our architecture is based on the M-net with a change: structure similarity…
We propose a fully automatic minutiae extractor, called MinutiaeNet, based on deep neural networks with compact feature representation for fast comparison of minutiae sets. Specifically, first a network, called CoarseNet, estimates the minutiae score map and minutiae orientation based on convolutional neural network and fingerprint domain knowledge (enhanced image, orientation field, and segmentation map). Subsequently, another network, called FineNet, refines the candidate minutiae locations based on score map. We demonstrate the effectiveness of using the fingerprint domain knowledge togethe…
Key Laboratory of Machine Perception (MOE), School of EECS, Peking University（北京大学信息科学技术学院机器感知与智能教育部重点实验室）
Minutiae extraction is of critical importance in automated fingerprint recognition. Previous works on rolled/slap fingerprints failed on latent fingerprints due to noisy ridge patterns and complex background noises. In this paper, we propose a new way to design deep convolutional network combining domain knowledge and the representation ability of deep learning. In terms of orientation estimation, segmentation, enhancement and minutiae extraction, several typical traditional methods performed well on rolled/slap fingerprints are transformed into convolutional manners and integrated as an unifi…
Given a full fingerprint image (rolled or slap), we present CycleGAN models to generate multiple latent impressions of the same identity as the full print. Our models can control the degree of distortion, noise, blurriness and occlusion in the generated latent print images to obtain Good, Bad and Ugly latent image categories as introduced in the NIST SD27 latent database. The contributions of our work are twofold: (i) demonstrate the similarity of synthetically generated latent fingerprint images to crime scene latents in NIST SD27 and MSP databases as evaluated by the NIST NFIQ 2 quality meas…
Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；Tongji University（同济大学）
Interactive segmentation has recently been explored to effectively and efficiently harvest high-quality segmentation masks by iteratively incorporating user hints. While iterative in nature, most existing interactive segmentation methods tend to ignore the dynamics of successive interactions and take each interaction independently. We here propose to model iterative interactive image segmentation with a Markov decision process (MDP) and solve it with reinforcement learning (RL) where each voxel is treated as an agent. Considering the large exploration space for voxel-wise prediction and the de…
Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning
Chinese University of Hong Kong, Hong Kong SAR, China（香港中文大学）；Hunyuan Group, Tencent（腾讯洪音集团）；Institute of Automation, the Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所）；Dalian University of Technology, Dalian, China（大连理工大学）；Stanford University, Stanford, USA（斯坦福大学）
Beijing University of Posts and Telecommunications（北京邮电大学）
Medical image segmentation is one of the important tasks of computer-aided diagnosis in medical image analysis. Since most medical images have the characteristics of blurred boundaries and uneven intensity distribution, through existing segmentation methods, the discontinuity within the target area and the discontinuity of the target boundary are likely to lead to rough or even erroneous boundary delineation. In this paper, we propose a new iterative refined interactive segmentation method for medical images based on agent reinforcement learning, which focuses on the problem of target segmenta…
Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）
Existing automatic 3D image segmentation methods usually fail to meet the clinic use. Many studies have explored an interactive strategy to improve the image segmentation performance by iteratively incorporating user hints. However, the dynamic process for successive interactions is largely ignored. We here propose to model the dynamic process of iterative interactive image segmentation as a Markov decision process (MDP) and solve it with reinforcement learning (RL). Unfortunately, it is intractable to use single-agent RL for voxel-wise prediction due to the large exploration space. To reduce …
New York University（纽约大学）；Huawei Technologies（华为技术有限公司）
Background/foreground segmentation has a lot of applications in image and video processing. In this paper, a segmentation algorithm is proposed which is mainly designed for text and line extraction in screen content. The proposed method makes use of the fact that the background in each block is usually smoothly varying and can be modeled well by a linear combination of a few smoothly varying basis functions, while the foreground text and graphics create sharp discontinuity. The algorithm separates the background and foreground pixels by trying to fit pixel values in the block into a smooth fun…
MSA2-Net: Utilizing Self-Adaptive Convolution Module to Extract Multi-Scale Information in Medical Image Segmentation
School of Artificial Intelligence, Nanning Normal University, Nanning, Guangxi, 530100, People's Republic of China（南宁师范学院人工智能学院）；School of Artificial Intelligence, Guangxi Minzu University, Nanning, Guangxi, 530300, People’s Republic of China（广西民族大学人工智能学院）
AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent
School of Mathematics, Shandong University（山东大学数学学院）；School of Information Science and Engineering, Shandong Normal University（山东师范大学信息科学与工程学院）；Yeshiva University（叶史瓦大学）；School of cyber science and technology, Sun Yat-sen University（中山大学网络空间安全学院）
Automatic segmentation in MR brain images is important for quantitative analysis in large-scale studies with images acquired at all ages. This paper presents a method for the automatic segmentation of MR brain images into a number of tissue classes using a convolutional neural network. To ensure that the method obtains accurate segmentation details as well as spatial consistency, the network uses multiple patch sizes and multiple convolution kernel sizes to acquire multi-scale information about each voxel. The method is not dependent on explicit features, but learns to recognise the informatio…
Understanding From Human Perspective: A Multi-agent System for Interactive Egocentric Medical Image Segmentation
School of Instrument Science and Engineering, Southeast University（东南大学仪器科学与工程学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Sichuan University-Pittsburgh Institute, Sichuan University（四川大学匹兹堡学院）；Department of Biomedical Engineering, Case Western Reserve University（凯斯西储大学生物医学工程系）
We present a new technique for deep reinforcement learning that automatically detects moving objects and uses the relevant information for action selection. The detection of moving objects is done in an unsupervised way by exploiting structure from motion. Instead of directly learning a policy from raw images, the agent first learns to detect and segment moving objects by exploiting flow information in video sequences. The learned representation is then used to focus the policy of the agent on the moving objects. Over time, the agent identifies which objects are critical for decision making an…
Shifting Adaptation from Weight Space to Memory Space: A Memory-Augmented Agent for Medical Image Segmentation
Department of Electrical and Computer Engineering, University of California, Santa Barbara, Santa Barbara, CA 93106, USA（电子工程系，加州大学圣芭芭拉分校）；College of Engineering, Northeastern University, Boston, MA 02115, USA（工程学院，东北大学）；Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA（生物医学工程系，新泽西理工学院）；United Imaging Intelligence, Burlington, MA 01803, USA（联合影像智能公司）；Department of Radiation Oncology, Mayo Clinic, Scottsdale, AZ 85259, USA（放射肿瘤科，梅奥诊所）；Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02115, USA（放射科，麻省总医院和哈佛医学院）；School of Computing, University of Georgia, Athens, GA 30602, USA（计算学院，佐治亚大学）
DuMeta++: Spatiotemporal Dual Meta-Learning for Generalizable Few-Shot Brain Tissue Segmentation Across Diverse Ages
School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；Key Laboratory of Biomedical Information Engineering of Ministry of Education, School of Life Science and Technology, Xi’an Jiaotong University（教育部生物医学信息工程重点实验室，西安交通大学生命科学与技术学院）
Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Saarland University（萨尔兰大学）
Segmentation of nanoscale electron microscopy (EM) images is crucial but still challenging in connectomics research. One reason for this is that none of the existing segmentation methods are error-free, so they require proofreading, which is typically implemented as an interactive, semi-automatic process via manual intervention. Herein, we propose a fully automatic proofreading method based on reinforcement learning that mimics the human decision process of detection, classification, and correction of segmentation errors. We systematically design the proposed system by combining multiple reinf…
Age related macular degeneration (AMD) is one of the major causes for blindness in the elderly population. In this report, we propose deep learning based methods for retinal analysis using color fundus images for computer aided diagnosis of AMD. We leverage the recent state of the art deep networks for building a single fundus image based AMD classification pipeline. We also propose methods for the other directly relevant and auxiliary tasks such as lesions detection and segmentation, fovea detection and optic disc segmentation. We propose the use of generative adversarial networks (GANs) for …
MedVeriSeg: Teaching LISA-Like Medical Segmentation Models to Verify Query Validity Without Extra Training
School of Aeronautics and Astronautics, Zhejiang University（浙江大学航空宇航学院）；Southern Medical University（南方医科大学）；School of Computer Science and Technology (School of Artificial Intelligence), Zhejiang Sci-Tech University（浙江科技学院计算机科学与技术学院（人工智能学院））；College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）
Automatic segmentation of brain MR images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) is critical for tissue volumetric analysis and cortical surface reconstruction. Due to dramatic structural and appearance changes associated with developmental and aging processes, existing brain tissue segmentation methods are only viable for specific age groups. Consequently, methods developed for one age group may fail for another. In this paper, we make the first attempt to segment brain tissues across the entire human lifespan (0-100 years of age) using a unified deep learning…
In this work, we introduce panoramic panoptic segmentation, as the most holistic scene understanding, both in terms of Field of View (FoV) and image-level understanding for standard camera-based input. A complete surrounding understanding provides a maximum of information to a mobile agent. This is essential information for any intelligent vehicle to make informed decisions in a safety-critical dynamic environment such as real-world traffic. In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers t…
