We present the first unsupervised deep learning method for pollen analysis using bright-field microscopy. Using a modest dataset of 650 images of pollen grains collected from honey, we achieve family level identification of pollen. We embed images of pollen grains into a low-dimensional latent space and compare Euclidean and Riemannian metrics on these spaces for clustering. We propose this system for automated analysis of pollen and other microscopic biological structures which have only small or unlabelled datasets available.
Million-scale multimodal pollen microscopy with expert-guided foundation models
Department of Physics of Complex Systems, ELTE Eötvös Loránd University（ELTE罗兰大学复杂物理系）；The Palynological Laboratory at the Swedish Museum of Natural History（瑞典自然历史博物馆孢粉学实验室）；National Centre for Public Health and Pharmacy（国家公共卫生与药品中心）；INRAE, UR 546 BioSP, Site Agroparc（法国国家农业、食品与环境研究院，UR 546 BioSP，阿格罗帕克园区）；National Korányi Institute for Pulmonology（国家科拉尼肺病研究所）；Health Data Science and AI Knowledge Centre, Health Services Management Training Centre, Faculty of Health and Public Administration, Semmelweis University（塞梅维什大学健康与公共管理学院卫生服务管理培训中心健康数据科学与人工智能知识中心）；Department of Biological Physics, ELTE Eötvös Loránd University（ELTE罗兰大学生物物理系）
提出百万级多模态花粉显微镜数据集Pollen AI Atlas，结合专家引导的视觉-语言模型生成形态描述，实现跨区域、跨设置的高精度花粉识别与检索。
We propose a robust approach for performing automatic species-level recognition of fossil pollen grains in microscopy images that exploits both global shape and local texture characteristics in a patch-based matching methodology. We introduce a novel criteria for selecting meaningful and discriminative exemplar patches. We optimize this function during training using a greedy submodular function optimization framework that gives a near-optimal solution with bounded approximation error. We use these selected exemplars as a dictionary basis and propose a spatially-aware sparse coding method to m…
Imperial College London（帝国理工学院）；University College London（伦敦大学学院）；Hampton Court House（汉普顿法院学校）
Honey has been collected and used by humankind as both a food and medicine for thousands of years. However, in the modern economy, honey has become subject to mislabelling and adulteration making it the third most faked food product in the world. The international scale of fraudulent honey has had both economic and environmental ramifications. In this paper, we propose a novel method of identifying fraudulent honey using machine learning augmented microscopy.
University of California, Los Angeles（加州大学洛杉矶分校）；California Nano Systems Institute (CNSI)（加州纳米系统研究所）；David Geffen School of Medicine（大卫·格芬医学院）
Deep learning brings bright-field microscopy contrast to holographic images of a sample volume, bridging the volumetric imaging capability of holography with the speckle- and artifact-free image contrast of bright-field incoherent microscopy.
University of South Bohemia in České Budějovice（南波希米亚大学）
This article presents an algorithm for the evaluation of organelles' movements inside of an unmodified live cell. We used a time-lapse image series obtained using wide-field bright-field photon transmission microscopy as an algorithm input. The benefit of the algorithm is the application of the Rényi information entropy, namely a variable called a point information gain, which enables to highlight the borders of the intracellular organelles and to localize the organelles' centers of mass with the precision of one pixel.
Detecting and segmenting individual cells from microscopy images is critical to various life science applications. Traditional cell segmentation tools are often ill-suited for applications in brightfield microscopy due to poor contrast and intensity heterogeneity, and only a small subset are applicable to segment cells in a cluster. In this regard, we introduce a novel supervised technique for cell segmentation in a multi-task learning paradigm. A combination of a multi-task loss, based on the region and cell boundary detection, is employed for an improved prediction efficiency of the network.…
Automated Palynological Analysis System: Integrating Deep Metric Learning, Detection and Classification in Bright Field Microscopy
Departamento de Física, Facultad de Ciencias Físicas y Matemáticas, Universidad de Concepción（物理系，物理与数学科学学院，康塞普西翁大学）；Facultad de Ingeniería, Universidad San Sebastián（工程学院，圣塞巴斯蒂安大学）；Facultad de Ingeniería, Universidad Andres Bello（工程学院，安德烈斯·贝洛大学）；Universidad Arturo Prat, Sede Victoria（阿尔托普大学，维多利亚校区）；Laboratory of Palynology and Plant Ecology, School of Sciences and Technologies, University of Concepción（古生物与植物生态实验室，科学与技术学院，康塞普西翁大学）
University of South Bohemia in České Budějovice（捷克布杰约维采南波希米亚大学）；Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences（亥姆霍兹波茨坦中心德国地球科学研究中心）
Living cell segmentation from bright-field light microscopy images is challenging due to the image complexity and temporal changes in the living cells. Recently developed deep learning (DL)-based methods became popular in medical and microscopy image segmentation tasks due to their success and promising outcomes. The main objective of this paper is to develop a deep learning, U-Net-based method to segment the living cells of the HeLa line in bright-field transmitted light microscopy. To find the most suitable architecture for our datasets, a residual attention U-Net was proposed and compared w…
Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Despite its broad availability, volumetric information acquisition from Bright-Field Microscopy (BFM) is inherently difficult due to the projective nature of the acquisition process. We investigate the prediction of 3D cell instances from a set of BFM Z-Stack images. We propose a novel two-stage weakly supervised method for volumetric instance segmentation of cells which only requires approximate cell centroids annotation. Created pseudo-labels are thereby refined with a novel refinement loss with Z-stack guidance. The evaluations show that our approach can generalize not only to BFM Z-Stack d…
Neural Field-Based 3D Surface Reconstruction of Microstructures from Multi-Detector Signals in Scanning Electron Microscopy
State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；Alibaba Group（阿里巴巴集团）；Analysis Center of Agriculture, Life and Environment Sciences, Zhejiang University（浙江大学农业、生命与环境科学分析中心）
The most realistic information about the transparent sample such as a live cell can be obtained only using bright-field light microscopy. At high-intensity pulsing LED illumination, we captured a primary 12-bit-per-channel (bpc) response from an observed sample using a bright-field microscope equipped with a high-resolution (4872x3248) image sensor. In order to suppress data distortions originating from the light interactions with elements in the optical path, poor sensor reproduction (geometrical defects of the camera sensor and some peculiarities of sensor sensitivity), we propose a spectros…
Deep-learning Assisted Detection and Quantification of (oo)cysts of Giardia and Cryptosporidium on Smartphone Microscopy Images
Nepal Applied Mathematics and Informatics Institute for research (NAAMII)（尼泊尔应用数学与信息学研究所（NAAMII））；Center for Analytical Sciences, Kathmandu Institute of Applied Sciences (KIAS)（加德满都应用科学学院分析科学中心）；Central Department of Chemistry, Tribhuvan University（特里布万大学化学系）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California NanoSystems Institute (CNSI)（加州纳米系统研究所）；David Geffen School of Medicine（大卫·格芬医学院）
Using a deep neural network, we demonstrate a digital staining technique, which we term PhaseStain, to transform quantitative phase images (QPI) of labelfree tissue sections into images that are equivalent to brightfield microscopy images of the same samples that are histochemically stained. Through pairs of image data (QPI and the corresponding brightfield images, acquired after staining) we train a generative adversarial network (GAN) and demonstrate the effectiveness of this virtual staining approach using sections of human skin, kidney and liver tissue, matching the brightfield microscopy …
Image-to-image reconstruction problems with free or inexpensive metadata in the form of class labels appear often in biological and medical image domains. Existing text-guided or style-transfer image-to-image approaches do not translate to datasets where additional information is provided as discrete classes. We introduce and implement a model which combines image-to-image and class-guided denoising diffusion probabilistic models. We train our model on a real-world dataset of microscopy images used for drug discovery, with and without incorporating metadata labels. By exploring the properties …
We demonstrate a deep learning-based offline autofocusing method, termed Deep-R, that is trained to rapidly and blindly autofocus a single-shot microscopy image of a specimen that is acquired at an arbitrary out-of-focus plane. We illustrate the efficacy of Deep-R using various tissue sections that were imaged using fluorescence and brightfield microscopy modalities and demonstrate snapshot autofocusing under different scenarios, such as a uniform axial defocus as well as a sample tilt within the field-of-view. Our results reveal that Deep-R is significantly faster when compared with standard …
Time-lapse fluorescence microscopy (TLFM) is an important and powerful tool in synthetic biological research. Modeling TLFM experiments based on real data may enable researchers to repeat certain experiments with minor effort. This thesis is a study towards deep learning-based modeling of TLFM experiments on the image level. The modeling of TLFM experiments, by way of the example of trapped yeast cells, is split into two tasks. The first task is to generate synthetic image data based on real image data. To approach this problem, a novel generative adversarial network, for conditionalized and u…
Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning
University of California, Los Angeles（加州大学洛杉矶分校）；California NanoSystems Institute (CNSI), University of California, Los Angeles（加州大学洛杉矶分校加州纳米系统研究所）；Fatih Sultan Mehmet Vakif University（法提赫·苏丹·穆罕默德·瓦克夫大学）；David Geffen School of Medicine at the University of California, Los Angeles（加州大学洛杉矶分校大卫·格芬医学院）；Hadassah Hebrew University Medical Center（哈达萨希伯来大学医学中心）；Keck School of Medicine, University of Southern California（南加州大学凯克医学院）
SegmentAnything helps microscopy images based automatic and quantitative organoid detection and analysis
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；King’s College London（伦敦国王学院）；Eindhoven University of Technology（埃因霍温理工大学）；Institute for Complex Molecular Systems (ICMS)（复杂分子系统研究所）；Imperial College London（帝国理工学院）
本文提出利用 SegmentAnything 分割单个类器官并计算多项形态学指标的自动化显微镜分析流程，在 iPSC 衍生 NE 类器官明场图像上验证其与人工检测测量高度一致。
Practical Guidelines for Cell Segmentation Models Under Optical Aberrations in Microscopy
The Fifth Affiliated Hospital of Wenzhou Medical University（温州医科大学附属第五医院）；Tsinghua University（清华大学）；Hangzhou Dianzi University（杭州电子科技大学）；Shandong University（山东大学）
Cell image analysis is crucial in Alzheimer's research to detect the presence of A$β$ protein inhibiting cell function. Deep learning speeds up the process by making only low-level data sufficient for fruitful inspection. We first found Unet is most suitable in augmented microscopy by comparing performance in multi-class semantics segmentation. We develop the augmented microscopy method to capture nuclei in a brightfield image and the transformer using Unet model to convert an input image into a sequence of topological information. The performance regarding Intersection-over-Union is consisten…
Going Beyond U-Net: Assessing Vision Transformers for Semantic Segmentation in Microscopy Image Analysis
Institute of Computer Science, University of Tartu（塔尔图大学计算机科学研究所）；Center for Artificial Intelligence, Technical University of Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用技术大学人工智能中心）；STACC
Unsupervised cell segmentation by fast Gaussian Processes
Department of Statistics and Applied Probability, University of California, Santa Barbara, CA（统计与应用概率系，加州大学圣芭芭拉分校）；Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA（电气与计算机工程系，加州大学圣芭芭拉分校）
PSL University（巴黎文理研究大学）；Mines Paris（巴黎矿业学院）；Centre for Computational Biology（计算生物学中心）；Institut Curie（居里研究所）；INSERM（法国国家健康与医学研究院）；University of Montpellier（蒙彼利埃大学）；IGH（人类遗传学研究所）；CNRS（法国国家科学研究中心）
Deep learning provides us with powerful methods to perform nucleus or cell segmentation with unprecedented quality. However, these methods usually require large training sets of manually annotated images, which are tedious and expensive to generate. In this paper we propose to use In Silico Labeling (ISL) as a pretraining scheme for segmentation tasks. The strategy is to acquire label-free microscopy images (such as bright-field or phase contrast) along fluorescently labeled images (such as DAPI or CellMask). We then train a model to predict the fluorescently labeled images from the label-free…
Purdue University（普渡大学）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Air Force Research Laboratory（美国空军研究实验室）
Many material and biological samples in scientific imaging are characterized by non-local repeating structures. These are studied using scanning electron microscopy and electron tomography. Sparse sampling of individual pixels in a 2D image acquisition geometry, or sparse sampling of projection images with large tilt increments in a tomography experiment, can enable high speed data acquisition and minimize sample damage caused by the electron beam. In this paper, we present an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the non-local redundanc…
