Lung cancer is highly lethal, emphasizing the critical need for early detection. However, identifying lung nodules poses significant challenges for radiologists, who rely heavily on their expertise for accurate diagnosis. To address this issue, computer-aided diagnosis (CAD) systems based on machine learning techniques have emerged to assist doctors in identifying lung nodules from computed tomography (CT) scans. Unfortunately, existing networks in this domain often suffer from computational complexity, leading to high rates of false negatives and false positives, limiting their effectiveness. To address these challenges, we present an innovative model that harnesses the strengths of both convolutional neural networks and vision transformers. Inspired by object detection in videos, we treat each 3D CT image as a video, individual slices as frames, and lung nodules as objects, enabling a time-series application. The primary objective of our work is to overcome hardware limitations during model training, allowing for efficient processing of 2D data while utilizing inter-slice information for accurate identification based on 3D image context. We validated the proposed network by applying a 10-fold cross-validation technique to the publicly available Lung Nodule Analysis 2016 dataset. Our proposed architecture achieves an average sensitivity criterion of 97.84% and a competition performance metrics (CPM) of 96.0% with few parameters. Comparative analysis with state-of-the-art advancements in lung nodule identification demonstrates the significant accuracy achieved by our proposed model.
Concordia University（康考迪亚大学）；Concordia Institute for Information Systems Engineering（康考迪亚信息系统工程研究所）；Sunnybrook Health Sciences Centre（桑尼布鲁克健康科学中心）；University of Toronto（多伦多大学）
The paper proposes a novel hybrid discovery Radiomics framework that simultaneously integrates temporal and spatial features extracted from non-thin chest Computed Tomography (CT) slices to predict Lung Adenocarcinoma (LUAC) malignancy with minimum expert involvement. Lung cancer is the leading cause of mortality from cancer worldwide and has various histologic types, among which LUAC has recently been the most prevalent. LUACs are classified as pre-invasive, minimally invasive, and invasive adenocarcinomas. Timely and accurate knowledge of the lung nodules malignancy leads to a proper treatme…
Harbin Institute of Technology（哈尔滨工业大学）；CVTE Research（视源股份研究院）；Southern University of Science and Technology（南方科技大学）；Guangdong University of Technology（广东工业大学）；CVTE（视源股份）
In the management of lung nodules, we are desirable to predict nodule evolution in terms of its diameter variation on Computed Tomography (CT) scans and then provide a follow-up recommendation according to the predicted result of the growing trend of the nodule. In order to improve the performance of growth trend prediction for lung nodules, it is vital to compare the changes of the same nodule in consecutive CT scans. Motivated by this, we screened out 4,666 subjects with more than two consecutive CT scans from the National Lung Screening Trial (NLST) dataset to organize a temporal dataset ca…
Eurecat, Centre Tecnòlogic de Catalunya（加泰罗尼亚技术中心Eurecat）；Universitat Pompeu Fabra（庞培法布拉大学）；Vall d’Hebron University Hospital（瓦尔德希伯伦大学医院）；ICREA（加泰罗尼亚研究与高级研究院）
Nodule malignancy assessment is a complex, time-consuming and error-prone task. Current clinical practice requires measuring changes in size and density of the nodule at different time-points. State of the art solutions rely on 3D convolutional neural networks built on pulmonary nodules obtained from single CT scan per patient. In this work, we propose a two-stream 3D convolutional neural network that predicts malignancy by jointly analyzing two pulmonary nodule volumes from the same patient taken at different time-points. Best results achieve 77% of F1-score in test with an increment of 9% an…
Shanghai Jiao Tong University（上海交通大学）；Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；AI Institute Shanghai Jiao Tong University（上海交通大学人工智能研究院）；Shanghai Chest Hospital（上海市胸科医院）；Shanghai General Hospital（上海市第一人民医院）
Follow-up serves an important role in the management of pulmonary nodules for lung cancer. Imaging diagnostic guidelines with expert consensus have been made to help radiologists make clinical decision for each patient. However, tumor growth is such a complicated process that it is difficult to stratify high-risk nodules from low-risk ones based on morphologic characteristics. On the other hand, recent deep learning studies using convolutional neural networks (CNNs) to predict the malignancy score of nodules, only provides clinicians with black-box predictions. To this end, we propose a unifie…
Eurecat Centre Tecnològic de Catalunya（加泰罗尼亚埃urecat技术中心）；Universitat Pompeu Fabra（庞培法布拉大学）；Vall d’Hebron University Hospital（瓦尔德希伯伦大学医院）；ICREA（加泰罗尼亚研究与高等研究院）
Lung cancer follow-up is a complex, error prone, and time consuming task for clinical radiologists. Several lung CT scan images taken at different time points of a given patient need to be individually inspected, looking for possible cancerogenous nodules. Radiologists mainly focus their attention in nodule size, density, and growth to assess the existence of malignancy. In this study, we present a novel method based on a 3D siamese neural network, for the re-identification of nodules in a pair of CT scans of the same patient without the need for image registration. The network was integrated …
Linköping University（林雪平大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Tianjin University（天津大学）
State-of-the-art transformer-based video instance segmentation (VIS) approaches typically utilize either single-scale spatio-temporal features or per-frame multi-scale features during the attention computations. We argue that such an attention computation ignores the multi-scale spatio-temporal feature relationships that are crucial to tackle target appearance deformations in videos. To address this issue, we propose a transformer-based VIS framework, named MS-STS VIS, that comprises a novel multi-scale spatio-temporal split (MS-STS) attention module in the encoder. The proposed MS-STS module …
STACT-Time: Spatio-Temporal Cross Attention for Cine Thyroid Ultrasound Time Series Classification
Medical \& Imaging Informatics, Department of Radiological Sciences, David Geffen School of Medicine, University of California, Los Angeles (UCLA), CA, USA Department of Bioengineering, UCLA, Los Angeles, CA, USA Department of Electrical；Computer Engineering, UCLA, Los Angeles, CA, USA Department of Radiological Sciences, UCLA, Los Angeles, CA, USA
Many video instance segmentation (VIS) methods partition a video sequence into individual frames to detect and segment objects frame by frame. However, such a frame-in frame-out (FiFo) pipeline is ineffective to exploit the temporal information. Based on the fact that adjacent frames in a short clip are highly coherent in content, we propose to extend the one-stage FiFo framework to a clip-in clip-out (CiCo) one, which performs VIS clip by clip. Specifically, we stack FPN features of all frames in a short video clip to build a spatio-temporal feature cube, and replace the 2D conv layers in the…
Mask Propagation for Efficient Video Semantic Segmentation
ZIP Lab, Monash University（莫纳什大学）；Baidu Inc.（百度公司）；ReLER, AAII, UTS（悉尼科技大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织）；Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）
Huazhong University of Science and Technology（华中科技大学）；Bytedance（字节跳动）
In this work, we present SeqFormer for video instance segmentation. SeqFormer follows the principle of vision transformer that models instance relationships among video frames. Nevertheless, we observe that a stand-alone instance query suffices for capturing a time sequence of instances in a video, but attention mechanisms shall be done with each frame independently. To achieve this, SeqFormer locates an instance in each frame and aggregates temporal information to learn a powerful representation of a video-level instance, which is used to predict the mask sequences on each frame dynamically. …
Peking University（北京大学）；Nanyang Technological University（南洋理工大学）；SenseTime Research（商汤科技研究院）；The Chinese University of Hong Kong（香港中文大学）；Shanghai AI Lab（上海人工智能实验室）
This paper presents Video K-Net, a simple, strong, and unified framework for fully end-to-end video panoptic segmentation. The method is built upon K-Net, a method that unifies image segmentation via a group of learnable kernels. We observe that these learnable kernels from K-Net, which encode object appearances and contexts, can naturally associate identical instances across video frames. Motivated by this observation, Video K-Net learns to simultaneously segment and track "things" and "stuff" in a video with simple kernel-based appearance modeling and cross-temporal kernel interaction. Despi…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Features learned from single radiologic images are unable to provide information about whether and how much a lesion may be changing over time. Time-dependent features computed from repeated images can capture those changes and help identify malignant lesions by their temporal behavior. However, longitudinal medical imaging presents the unique challenge of sparse, irregular time intervals in data acquisition. While self-attention has been shown to be a versatile and efficient learning mechanism for time series and natural images, its potential for interpreting temporal distance between sparse,…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
The field of lung nodule detection and cancer prediction has been rapidly developing with the support of large public data archives. Previous studies have largely focused on cross-sectional (single) CT data. Herein, we consider longitudinal data. The Long Short-Term Memory (LSTM) model addresses learning with regularly spaced time points (i.e., equal temporal intervals). However, clinical imaging follows patient needs with often heterogeneous, irregular acquisitions. To model both regular and irregular longitudinal samples, we generalize the LSTM model with the Distanced LSTM (DLSTM) for tempo…
Reliability-Aware Weighted Multi-Scale Spatio-Temporal Maps for Heart Rate Monitoring
CVPR Unit, Indian Statistical Institute（印度统计研究所计算机视觉与模式识别单元）；Department of Computational and Data Sciences, Indian Institute of Science（印度科学研究所计算与数据科学系）
Research on video frame interpolation has made significant progress in recent years. However, existing methods mostly use off-the-shelf metrics to measure the quality of interpolation results with the exception of a few methods that employ user studies, which is time-consuming. As video frame interpolation results often exhibit unique artifacts, existing quality metrics sometimes are not consistent with human perception when measuring the interpolation results. Some recent deep learning-based perceptual quality metrics are shown more consistent with human judgments, but their performance on vi…
Amity School of Engineering and Technology, Amity University（阿米提大学阿米提工程技术学院）；Dr. Ram Manohar Lohia Awadh University（拉姆·马诺哈尔·洛希亚博士阿瓦德大学）
Computer Aided Diagnosis has emerged as an indispensible technique for validating the opinion of radiologists in CT interpretation. This paper presents a deep 3D Convolutional Neural Network (CNN) architecture for automated CT scan-based lung cancer detection system. It utilizes three dimensional spatial information to learn highly discriminative 3 dimensional features instead of 2D features like texture or geometric shape whick need to be generated manually. The proposed deep learning method automatically extracts the 3D features on the basis of spatio-temporal statistics.The developed model …
Computed tomography (CT) imaging could be very practical for diagnosing various diseases. However, the nature of the CT images is even more diverse since the resolution and number of the slices of a CT scan are determined by the machine and its settings. Conventional deep learning models are hard to tickle such diverse data since the essential requirement of the deep neural network is the consistent shape of the input data. In this paper, we propose a novel, effective, two-step-wise approach to tickle this issue for COVID-19 symptom classification thoroughly. First, the semantic feature embedd…
NORCE Norwegian Research Centre（NORCE挪威研究中心）；Hong Kong University of Science and Technology（香港科技大学）
Visual (image, video) quality assessments can be modelled by visual features in different domains, e.g., spatial, frequency, and temporal domains. Perceptual mechanisms in the human visual system (HVS) play a crucial role in generation of quality perception. This paper proposes a general framework for no-reference visual quality assessment using efficient windowed transformer architectures. A lightweight module for multi-stage channel attention is integrated into Swin (shifted window) Transformer. Such module can represent appropriate perceptual mechanisms in image quality assessment (IQA) to …
University of Padova（帕多瓦大学）；National Centre of Neurology and Psychiatry (NCNP)（国立精神·神经医疗研究中心）；Newcastle University（纽卡斯尔大学）；Newcastle upon Tyne Hospitals NHS Foundation Trust（纽卡斯尔医院NHS基金会信托）
Wearable technology for the automatic detection of gait events has recently gained growing interest, enabling advanced analyses that were previously limited to specialist centres and equipment (e.g., instrumented walkway). In this study, we present a novel method based on dilated convolutions for an accurate detection of gait events (initial and final foot contacts) from wearable inertial sensors. A rich dataset has been used to validate the method, featuring 71 people with Parkinson's disease (PD) and 67 healthy control subjects. Multiple sensors have been considered, one located on the fifth…
