Recognizing human actions from untrimmed videos is an important task in activity understanding, and poses unique challenges in modeling long-range temporal relations. Recent works adopt a predict-and-refine strategy which converts an initial prediction to action segments for global context modeling. However, the generated segment representations are often noisy and exhibit inaccurate segment boundaries, over-segmentation and other problems. To deal with these issues, we propose an attention based approach which we call \textit{temporal segment transformer}, for joint segment relation modeling and denoising. The main idea is to denoise segment representations using attention between segment and frame representations, and also use inter-segment attention to capture temporal correlations between segments. The refined segment representations are used to predict action labels and adjust segment boundaries, and a final action segmentation is produced based on voting from segment masks. We show that this novel architecture achieves state-of-the-art accuracy on the popular 50Salads, GTEA and Breakfast benchmarks. We also conduct extensive ablations to demonstrate the effectiveness of different components of our design.
Fully Transformer Networks for Semantic Image Segmentation
Institute of Deep Learning, Baidu Research(百度研究院深度学习研究所);National Engineering Laboratory for Deep Learning Technology and Application(深度学习技术及应用国家工程实验室);School of Software, XinJiang University(新疆大学软件学院)
Refined-Segmentation R-CNN: A Two-stage Convolutional Neural Network for Punctate White Matter Lesion Segmentation in Preterm Infants
Xidian University(西安电子科技大学);The First Affiliated Hospital of Xi'an Jiaotong University(西安交通大学第一附属医院);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)
利用自动算法准确分割婴儿大脑点状白质病变(PWML)可以降低出生后发育的潜在风险。如何有效地分割PWML已成为近年来医学图像分割领域的研究热点之一。在本文中,我们构建了一个有效的两阶段PWML语义分割网络的基础上病变的特点,称为细化分割R-CNN(RS RCNN)。我们提出了一种启发式RPN(H-RPN),它可以利用周围的PWML启发式分割的周围信息。此外,我们还设计了一个轻量级的分割网络,以快速的方式分割病变。采用稠密连接条件随机场(DCRF)对分割结果进行优化。我们仅使用T1 w MRI分割PWML。实验结果表明,该模型可以很好地分割出普通大小甚至像素大小的病变。Dice相似系数为0.6616,敏感度为0.7069,特异度为0.9997,Hausdorff距离为52.9130。所提出的方法优于国家的最先进的算法。(The本文的代码可在https://github.com/YalongLiu/Refined-Segmentation-R-CNN上获得)
SQA-SAM: Segmentation Quality Assessment for Medical Images Utilizing the Segment Anything Model
Nanjing University of Science and Technology(南京理工大学);Digital Medical Research Center, School of Basic Medical Sciences, Fudan University(复旦大学基础医学院数字医学研究中心);The Chinese University of Hong Kong(香港中文大学);University of Notre Dame(圣母大学)