State-of-the-art object detection and segmentation methods for microscopy images rely on supervised machine learning, which requires laborious manual annotation of training data. Here we present a self-supervised method based on time arrow prediction pre-training that learns dense image representations from raw, unlabeled live-cell microscopy videos. Our method builds upon the task of predicting the correct order of time-flipped image regions via a single-image feature extractor followed by a time arrow prediction head that operates on the fused features. We show that the resulting dense representations capture inherently time-asymmetric biological processes such as cell divisions on a pixel-level. We furthermore demonstrate the utility of these representations on several live-cell microscopy datasets for detection and segmentation of dividing cells, as well as for cell state classification. Our method outperforms supervised methods, particularly when only limited ground truth annotations are available as is commonly the case in practice. We provide code at https://github.com/weigertlab/tarrow.
Self-supervised Representation Learning for Cell Event Recognition through Time Arrow Prediction
Institute for Mathematical Innovation, University of Bath(巴斯大学数学创新研究所);Department of Computer Science, University of Bath(巴斯大学计算机科学系);Department of Life Sciences, University of Bath(巴斯大学生命科学系)
Trackastra: Transformer-based cell tracking for live-cell microscopy
Institute of Bioengineering, School of Life Sciences, EPFL(EPFL生命科学学院生物工程研究所);Center for Scalable Data Analytics and AI (ScaDS.AI)(可扩展数据分析与人工智能中心(ScaDS.AI));Faculty of Computer Science, TU Dresden(德累斯顿工业大学计算机科学学院)
Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm
Graduate School of Science and Technology, Nara Institute of Science and Technology(奈良科学技术大学研究生院);Bioresource Engineering Laboratory, Ishikawa Prefectural University(石川县立大学生物资源工程实验室);Department of Engineering Informatics, Osaka Electro-Communication University(大阪电讯大学工程信息学系);Exploratory Research Center on Life and Living Systems, National Institutes of Natural Sciences(国家自然科学研究院生命与生活系统探索研究中心);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院)
DART: A design-aware microfluidic chip paradigm for real-time live-cell image analysis
Institute for Bio- and Geosciences, IBG-1: Biotechnology(生物与地质科学研究所,IBG-1:生物技术);Computational Systems Biotechnology (AVT.CSB), RWTH Aachen University(计算系统生物技术(AVT.CSB),亚琛工业大学);Institute for Advanced Simulation, IAS-8: Data Analytics and Machine Learning(先进模拟研究所,IAS-8:数据分析与机器学习)
Fast light-field 3D microscopy with out-of-distribution detection and adaptation through Conditional Normalizing Flows
Technical University of Munich(慕尼黑工业大学);Munich Institute of Biomedical Engineering(慕尼黑生物医学工程研究所);Massachusetts Institute of Technology(麻省理工学院);University of Bern(伯尔尼大学)
Self-supervised Pretraining of Cell Segmentation Models
School of Computer Science, Georgia Institute of Technology(佐治亚理工学院计算机科学学院);School of Materials Science and Engineering, Georgia Institute of Technology(佐治亚理工学院材料科学与工程学院);Department of Biomedical Engineering, Georgia Institute of Technology(佐治亚理工学院生物医学工程系);School of Mechanical Engineering, Georgia Institute of Technology(佐治亚理工学院机械工程学院)
ML-SIM: A deep neural network for reconstruction of structured illumination microscopy images
University of Cambridge(剑桥大学);Department of Chemical Engineering and Biotechnology(化学工程与生物技术系);Department of Computer Science and Technology(计算机科学与技术系)
EAP4EMSIG -- Enhancing Event-Driven Microscopy for Microfluidic Single-Cell Analysis
Institute for Automation and Applied Informatics (IAI), Karlsruhe Institute of Technology(自动化与应用信息研究所,卡尔斯鲁厄理工学院);Institute of Biological and Chemical Systems (IBCS), Karlsruhe Institute of Technology(生物与化学系统研究所,卡尔斯鲁厄理工学院);Institute for Data Science and Machine Learning (IAS-8), Forschungszentrum Jülich GmbH(数据科学与机器学习研究所,吕内堡研究中心 GmbH);Institute of Bio- and Geosciences (IBG-1), Forschungszentrum Jülich GmbH(生物与地球科学研究所,吕内堡研究中心 GmbH);Computational Systems Biology (AVT-CSB), RWTH Aachen University(计算系统生物学(AVT-CSB),亚琛工业大学)