DM-QPMNET: Dual-modality fusion network for cell segmentation in quantitative phase microscopy
College of Computing and Informatics, University of North Carolina at Charlotte, NC, USA（计算与信息学院，北卡罗来纳大学夏洛特分校）；Department of Physics and Optical Science, University of North Carolina at Charlotte, NC, USA（物理与光学科学系，北卡罗来纳大学夏洛特分校）；Center for TAIMing AI, University of North Carolina at Charlotte, NC, USA（TAIMing AI中心，北卡罗来纳大学夏洛特分校）
Rajatsubhra Chakraborty, Ana Espinosa-Momox, Riley Haskin, Depeng Xu, Rosario Porras-Aguilar
Cell segmentation in single-shot quantitative phase microscopy (ssQPM) faces challenges from traditional thresholding methods that are sensitive to noise and cell density, while deep learning approaches using simple channel concatenation fail to exploit the complementary nature of polarized intensity images and phase maps. We introduce DM-QPMNet, a dual-encoder network that treats these as distinct modalities with separate encoding streams. Our architecture fuses modality-specific features at intermediate depth via multi-head attention, enabling polarized edge and texture representations to selectively integrate complementary phase information. This content-aware fusion preserves training stability while adding principled multi-modal integration through dual-source skip connections and per-modality normalization at minimal overhead. Our approach demonstrates substantial improvements over monolithic concatenation and single-modality baselines, showing that modality-specific encoding with learnable fusion effectively exploits ssQPM's simultaneous capture of complementary illumination and phase cues for robust cell segmentation.
Cell segmentation is a fundamental task for computational biology analysis. Identifying the cell instances is often the first step in various downstream biomedical studies. However, many cell segmentation algorithms, including the recently emerging deep learning-based methods, still show limited generality under the multi-modality environment. Weakly Supervised Cell Segmentation in Multi-modality High-Resolution Microscopy Images was hosted at NeurIPS 2022 to tackle this problem. We propose MEDIAR, a holistic pipeline for cell instance segmentation under multi-modality in this challenge. MEDIA…
National University of Defense Technology（国防科技大学）；Wuhan University（武汉大学）
Self-supervised pre-training has become the priory choice to establish reliable neural networks for automated recognition of massive biomedical microscopy images, which are routinely annotation-free, without semantics, and without guarantee of quality. Note that this paradigm is still at its infancy and limited by closely related open issues: 1) how to learn robust representations in an unsupervised manner from unlabelled biomedical microscopy images of low diversity in samples? and 2) how to obtain the most significant representations demanded by a high-quality segmentation? Aiming at these i…
The Multi-modality Cell Segmentation Challenge: Towards Universal Solutions
Peter Munk Cardiac Centre, University Health Network（大学健康网络彼得·蒙克心脏中心）；Department of Laboratory Medicine and Pathobiology, University of Toronto（多伦多大学实验医学与病理生物学系）；Vector Institute（向量研究所）；Department of Molecular Genetics, University of Toronto（多伦多大学分子遗传学系）；Donnelly Centre, University of Toronto（多伦多大学唐纳利中心）；Princess Margaret Cancer Centre, University Health Network（大学健康网络玛格丽特公主癌症中心）；School of Medicine and Pharmacy, Ocean University of China（中国海洋大学医药学院）；Laboratory Oncology, Dr. BRA-IRCH, All India Institute of Medical Sciences（全印度医学科学研究所）；Department of Image Reconstruction, Nanjing Anke Medical Technology Co., Ltd.（南京安科医疗科技有限公司）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Graduate School of AI, KAIST（韩国科学技术院人工智能研究生院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Chinese University of Hongkong (Shenzhen)（香港中文大学（深圳））
University of California, Los Angeles（加州大学洛杉矶分校）；California Nano Systems Institute (CNSI)（加州纳米系统研究所）；David Geffen School of Medicine（大卫·格芬医学院）
Deep learning brings bright-field microscopy contrast to holographic images of a sample volume, bridging the volumetric imaging capability of holography with the speckle- and artifact-free image contrast of bright-field incoherent microscopy.
ETH Zurich（苏黎世联邦理工学院）；University Hospital（大学医院）；University of Zurich（苏黎世大学）；Uppsala University（乌普萨拉大学）
Fluorescence microscopy images contain several channels, each indicating a marker staining the sample. Since many different marker combinations are utilized in practice, it has been challenging to apply deep learning based segmentation models, which expect a predefined channel combination for all training samples as well as at inference for future application. Recent work circumvents this problem using a modality attention approach to be effective across any possible marker combination. However, for combinations that do not exist in a labeled training dataset, one cannot have any estimation of…
Technische Universität München（慕尼黑工业大学）；Helmholtz Zentrum München（亥姆霍兹慕尼黑研究中心）；TranslaTUM Center for Translational Cancer Research（TranslaTUM转化癌症研究中心）；University of Zürich（苏黎世大学）
Novel multimodal imaging methods are capable of generating extensive, super high resolution datasets for preclinical research. Yet, a massive lack of annotations prevents the broad use of deep learning to analyze such data. So far, existing generative models fail to mitigate this problem because of frequent labeling errors. In this paper, we introduce a novel generative method which leverages real anatomical information to generate realistic image-label pairs of tumours. We construct a dual-pathway generator, for the anatomical image and label, trained in a cycle-consistent setup, constrained …
Harvard University（哈佛大学）；Harvard Medical School（哈佛医学院）；Boston Children’s Hospital（波士顿儿童医院）；ESRF, The European Synchrotron（欧洲同步辐射装置）
Comprehensive, synapse-resolution imaging of the brain will be crucial for understanding neuronal computations and function. In connectomics, this has been the sole purview of volume electron microscopy (EM), which entails an excruciatingly difficult process because it requires cutting tissue into many thin, fragile slices that then need to be imaged, aligned, and reconstructed. Unlike EM, hard X-ray imaging is compatible with thick tissues, eliminating the need for thin sectioning, and delivering fast acquisition, intrinsic alignment, and isotropic resolution. Unfortunately, current state-of-…
ETH Zurich（苏黎世联邦理工学院）；University Hospital and University of Zurich（苏黎世大学医院和苏黎世大学）；Uppsala University（乌普萨拉大学）
Fluorescence microscopy allows for a detailed inspection of cells, cellular networks, and anatomical landmarks by staining with a variety of carefully-selected markers visualized as color channels. Quantitative characterization of structures in acquired images often relies on automatic image analysis methods. Despite the success of deep learning methods in other vision applications, their potential for fluorescence image analysis remains underexploited. One reason lies in the considerable workload required to train accurate models, which are normally specific for a given combination of markers…
University of California, Davis（加州大学戴维斯分校）；University of San Francisco Data Institute（旧金山大学数据研究所）；Wicklow AI in Medicine Research Initiative（威克洛医学人工智能研究计划）；University of Texas, San Antonio Health（德克萨斯大学圣安东尼奥健康分校）
MUSE is a novel slide-free imaging technique for histological examination of tissues that can serve as an alternative to traditional histology. In order to bridge the gap between MUSE and traditional histology, we aim to convert MUSE images to resemble authentic hematoxylin- and eosin-stained (H&E) images. We evaluated four models: a non-machine-learning-based color-mapping unmixing-based tool, CycleGAN, DualGAN, and GANILLA. CycleGAN and GANILLA provided visually compelling results that appropriately transferred H&E style and preserved MUSE content. Based on training an automated critic on re…
SIMPLER: H&E-Informed Representation Learning for Structured Illumination Microscopy
Kahlert School of Computing, University of Utah, Salt Lake City, UT 84112, USA Scientific Computing \& Imaging Institute, University of Utah, Salt Lake City, UT 84112, USA Instapath Inc., Houston, TX 77021 Tulane University, Department of Biomedical Engineering, New Orleans, Louisiana, United States
École Polytechnique de Montréal（蒙特利尔理工学院）；Montreal Institute for Learning Algorithms（蒙特利尔学习算法研究所）；Imagia Inc.（Imagia公司）；Université de Montréal（蒙特利尔大学）；CHUM Research Center（CHUM研究中心）
In this paper, we introduce a simple, yet powerful pipeline for medical image segmentation that combines Fully Convolutional Networks (FCNs) with Fully Convolutional Residual Networks (FC-ResNets). We propose and examine a design that takes particular advantage of recent advances in the understanding of both Convolutional Neural Networks as well as ResNets. Our approach focuses upon the importance of a trainable pre-processing when using FC-ResNets and we show that a low-capacity FCN model can serve as a pre-processor to normalize medical input data. In our image segmentation pipeline, we use …
Deep learning denoising unlocks quantitative insights in operando materials microscopy
Department of Chemical Engineering（化学工程系）；Massachusetts Institute of Technology（麻省理工学院）；Department of Energy, Environmental and Chemical Engineering（能源、环境与化学工程系）；Washington University in St. Louis（圣路易斯华盛顿大学）；Research Institute（研究所）；University of Alabama in Huntsville（阿拉巴马大学亨茨维尔分校）；Department of Mechanical and Aerospace Engineering（机械与航空航天工程系）；Institute of Materials Science and Engineering（材料科学与工程研究所）；Department of Mathematics（数学系）
HASSL: Hierarchy-Aware Self-Supervised Learning Framework for Single Cell Microscopy
TUM.ai, Munich, Germany（慕尼黑工业大学人工智能中心，德国慕尼黑）；Ludwig-Maximilian-University, Munich, Germany（慕尼黑大学，德国慕尼黑）；Technical University of Munich, Munich, Germany（慕尼黑工业大学，德国慕尼黑）；Institute of Stem Cell Research, Helmholtz Munich, German Research Center for Environmental Health, Neuherberg, Germany（干细胞研究所，亥姆霍兹慕尼黑中心，德国环境健康研究中心，德国新赫伯格）；Computational Health Center, Helmholtz Munich - German Research Center for Environmental Health, Neuherberg, Germany（计算健康中心，亥姆霍兹慕尼黑中心 - 德国环境健康研究中心，德国新赫伯格）；Department of Medicine III, Ludwig-Maximilian-University Hospital, Munich, Germany（医学部III，慕尼黑大学医院，德国慕尼黑）；Department of Physics, Ludwig-Maximilian-University, Munich, Germany（物理系，慕尼黑大学，德国慕尼黑）；German Cancer Consortium (DKTK), partner site Munich, Germany（德国癌症联盟（DKTK），慕尼黑合作站点，德国慕尼黑）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML），德国慕尼黑）
A Multimodal 3D Foundation Model for Light Sheet Fluorescence Microscopy Enables Few-Shot Segmentation, Classification, and Deblurring
Tri-Institutional Program in Computational Biology \& Medicine, Weill Cornell Medicine, New York, NY, USA Department of Radiology, Weill Cornell Medicine, New York, NY, USA Helen；Robert Appel Alzheimers Disease Research Institute, Feil Family Brain；Mind Research Institute, Weill Cornell Medicine, New York, NY, USA Graduate Program in Physiology, Biophysics；Systems Biology, Weill Cornell Medicine, New York, NY, USA Cornell Tech, New York, NY, USA Institute for Intelligent Biotechnologies (iBIO), Helmholtz Center Munich, Neuherberg, Germany Institute for Stroke；Dementia Research, Klinikum der Universität München, Ludwig-Maximilians University Munich, Munich, Germany
AxonSynth: Domain-Randomized Synthetic Data for Zero-Shot 3D Axon Segmentation in Light-Sheet Microscopy
University of Massachusetts Boston（马萨诸塞大学波士顿分校）；Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School（阿西诺拉·A·马蒂诺斯生物医学成像中心，马萨诸塞总医院和哈佛医学院）；Universidad San Sebastián（圣塞巴斯蒂安大学）；Université Claude Bernard Lyon 1, Université de Lyon（里昂第一大学，里昂大学）；Columbia University in the City of New York（纽约市哥伦比亚大学）；University College London（伦敦大学学院）
$μ$Match: Foundation Models for Semi-supervised Learning and Domain Adaptation in EM
Life and Medical Sciences Institute (LIMES), University of Bonn, Germany（波恩大学生命与医学科学研究所（LIMES））；Institute of Computer Science, Georg-August-University Göttingen, Germany（哥廷根大学计算机科学研究所）
Wide-field lensfree on-chip microscopy, which leverages holography principles to capture interferometric light-field encodings without lenses, is an emerging imaging modality with widespread interest given the large field-of-view compared to lens-based techniques. In this study, we introduce the idea of laser light-field fusion for lensfree on-chip phase contrast nanoscopy, where interferometric laser light-field encodings acquired using an on-chip setup with laser pulsations at different wavelengths are fused to produce marker-free phase contrast images of superior quality with resolving powe…
