TASL-Net:用于双模态超声视频智能诊断的三重注意力选择性学习网络
TASL-Net: Tri-Attention Selective Learning Network for Intelligent Diagnosis of Bimodal Ultrasound Video
AI总结:
本文提出 TASL-Net,通过时间-强度曲线视频选择器、最早增强位置检测器和卷积-transformer 互惠编码策略,将超声医师的三类诊断注意力嵌入双模态超声视频智能诊断,并在肺、乳腺和肝脏数据集上验证了其优越性能。
摘要:
在双模态(灰阶与造影增强)超声视频的智能诊断中,医学领域知识——例如超声医师浏览视频的方式、他们重点关注的区域以及特别留意的特征——对于实现精准诊断具有决定性作用。将医学知识嵌入深度学习网络,不仅能提升性能,还能增强网络的临床可信度与可靠性。然而,如何自动聚焦视频中这些因人而异、因病而异的特征,并使网络能够全面且高效地编码双模态信息,仍是一个棘手的挑战。本文提出了一种新颖的三重注意力选择性学习网络(TASL-Net)来应对这一挑战,并将超声医师的三类诊断注意力自动嵌入到一个互惠 transformer 框架中,用于双模态超声视频的智能诊断。首先,设计了一个基于时间-强度曲线的视频选择器,以模拟超声医师的时间注意力,从而在提高 TASL-Net 计算效率的同时去除大量冗余信息。然后,为在造影增强视频分析中引入超声医师的空间注意力,我们提出了基于结构相似性变化的最早增强位置检测器,使 TASL-Net 聚焦于病灶内外灌注变化的差异。最后,通过提出一种结合卷积与 transformer 的互惠编码策略,TASL-Net 具备了对灰阶视频结构特征和造影增强视频灌注变化的双模态注意力。这些模块协同工作,带来了优越的性能。我们在肺、乳腺和肝脏三个数据集上对 TASL-Net 的性能进行了详细的实验验证。
英文摘要:
In the intelligent diagnosis of bimodal (gray-scale and contrast-enhanced) ultrasound videos, medical domain knowledge such as the way sonographers browse videos, the particular areas they emphasize, and the features they pay special attention to, plays a decisive role in facilitating precise diagnosis. Embedding medical knowledge into the deep learning network can not only enhance performance but also boost clinical confidence and reliability of the network. However, it is an intractable challenge to automatically focus on these person- and disease-specific features in videos and to enable networks to encode bimodal information comprehensively and efficiently. This paper proposes a novel Tri-Attention Selective Learning Network (TASL-Net) to tackle this challenge and automatically embed three types of diagnostic attention of sonographers into a mutual transformer framework for intelligent diagnosis of bimodal ultrasound videos. Firstly, a time-intensity-curve-based video selector is designed to mimic the temporal attention of sonographers, thus removing a large amount of redundant information while improving computational efficiency of TASL-Net. Then, to introduce the spatial attention of the sonographers for contrast-enhanced video analysis, we propose the earliest-enhanced position detector based on structural similarity variation, on which the TASL-Net is made to focus on the differences of perfusion variation inside and outside the lesion. Finally, by proposing a mutual encoding strategy that combines convolution and transformer, TASL-Net possesses bimodal attention to structure features on gray-scale videos and to perfusion variations on contrast-enhanced videos. These modules work collaboratively and contribute to superior performance. We conduct a detailed experimental validation of TASL-Net's performance on three datasets, including lung, breast, and liver.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。