一种多模态医学影像交互框架
A Framework for Multimodal Medical Image Interaction
AI总结：
针对传统医学影像单模态交互的局限，提出基于虚拟现实的MMII框架，通过模型驱动的视听反馈提升解剖结构感知与肿瘤定位精度，经用户研究验证了其可学习性与临床应用潜力。
摘要：
医生依赖磁共振成像（MRI）等人体解剖学影像，在诊断和治疗过程中定位患者体内的感兴趣区域。尽管医学影像技术不断进步，信息传递仍为单模态。这种视觉呈现无法捕捉与人体组织进行真实多感官交互的复杂性。然而，实时感知患者解剖结构和疾病的多模态信息，对医疗操作的成功和患者预后至关重要。我们提出了多模态医学影像交互（Multimodal Medical Image Interaction，MMII）框架，让医学专家能够在三维空间中与人体组织进行动态的视听交互。在虚拟现实环境中，用户会接收符合物理规律的视听反馈，以提升对解剖结构的空间感知能力。MMII采用基于模型的声化（sonification）方法，根据组织的几何形状和物理属性生成声音，从而无需手工设计音效。我们开展了两项用户研究，共纳入34名普通专家和9名临床专家，以评估所提出交互框架的可学习性、可用性和准确性。结果显示，视听对应关系的可学习性极佳，因为在研究过程中正确关联率显著提升（p < 0.001）。与传统医学影像交互方式相比，MMII实现了更优的脑肿瘤定位准确率（p < 0.05）。我们的研究结果证实了这一新型框架增强医学影像交互的潜力，例如可应用于需要即时精准反馈的外科手术等场景。
英文摘要：
Medical doctors rely on images of the human anatomy, such as magnetic resonance imaging (MRI), to localize regions of interest in the patient during diagnosis and treatment. Despite advances in medical imaging technology, the information conveyance remains unimodal. This visual representation fails to capture the complexity of the real, multisensory interaction with human tissue. However, perceiving multimodal information about the patient's anatomy and disease in real-time is critical for the success of medical procedures and patient outcome. We introduce a Multimodal Medical Image Interaction (MMII) framework to allow medical experts a dynamic, audiovisual interaction with human tissue in three-dimensional space. In a virtual reality environment, the user receives physically informed audiovisual feedback to improve the spatial perception of anatomical structures. MMII uses a model-based sonification approach to generate sounds derived from the geometry and physical properties of tissue, thereby eliminating the need for hand-crafted sound design. Two user studies involving 34 general and nine clinical experts were conducted to evaluate the proposed interaction framework's learnability, usability, and accuracy. Our results showed excellent learnability of audiovisual correspondence as the rate of correct associations significantly improved (p < 0.001) over the course of the study. MMII resulted in superior brain tumor localization accuracy (p < 0.05) compared to conventional medical image interaction. Our findings substantiate the potential of this novel framework to enhance interaction with medical images, for example, during surgical procedures where immediate and precise feedback is needed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
