基于GNN辅助和对比学习的药物处方匹配新方法
A Novel Approach for Pill-Prescription Matching with GNN Assistance and Contrastive Learning
摘要：
用药错误是可能给患者造成不可预测后果的风险之一。为了减轻这种风险，我们开发了一种自动系统，可以正确识别移动图像中的药丸处方。具体来说，我们定义了一个所谓的药丸处方匹配任务，它试图将服用的药丸的图像与处方中的药丸名称相匹配。然后，我们提出了PIMA，一种新的方法，使用图神经网络（GNN）和对比学习来解决目标问题。特别是，GNN用于学习处方中文本框之间的空间相关性，从而突出显示携带药丸名称的文本框。此外，对比学习是用来促进跨模态的相似性建模之间的文本表示的药丸名称和视觉表示的药丸图像。我们进行了广泛的实验，并证明PIMA在我们构建的药丸和处方图像的真实数据集上优于基线模型。具体而言，与其他基线相比，PIMA将准确率从19.09%提高到46.95%。我们相信，我们的工作可以开辟新的机会，建立新的临床应用，提高药物安全性和病人护理。
英文摘要：
Medication mistaking is one of the risks that can result in unpredictable consequences for patients. To mitigate this risk, we develop an automatic system that correctly identifies pill-prescription from mobile images. Specifically, we define a so-called pill-prescription matching task, which attempts to match the images of the pills taken with the pills' names in the prescription. We then propose PIMA, a novel approach using Graph Neural Network (GNN) and contrastive learning to address the targeted problem. In particular, GNN is used to learn the spatial correlation between the text boxes in the prescription and thereby highlight the text boxes carrying the pill names. In addition, contrastive learning is employed to facilitate the modeling of cross-modal similarity between textual representations of pill names and visual representations of pill images. We conducted extensive experiments and demonstrated that PIMA outperforms baseline models on a real-world dataset of pill and prescription images that we constructed. Specifically, PIMA improves the accuracy from 19.09% to 46.95% compared to other baselines. We believe our work can open up new opportunities to build new clinical applications and improve medication safety and patient care.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
