医学知识图谱辅助下基于图像的上下文药丸识别
Image-based Contextual Pill Recognition with Medical Knowledge Graph Assistance
摘要：
根据在各种条件和背景下捕获的图像识别药丸已经变得越来越重要。在文献中，已经有几项努力致力于利用基于深度学习的方法来解决药丸识别问题。然而，由于药丸外观之间的高度相似性，经常发生误识别，这给药丸识别带来了挑战。为此，在本文中，我们介绍了一种名为PIKA的新方法，该方法利用外部知识来提高药丸识别的准确性。具体来说，我们解决了一个实际的情况下（我们称之为上下文药丸识别），旨在识别药丸在病人的药丸摄入量的图片。首先，我们提出了一种新的方法，用于在存在外部数据源（在这种情况下，处方）的情况下对药丸之间的隐式关联进行建模。其次，我们提出了一个基于行走的图嵌入模型，将图空间转换到向量空间，提取药丸的浓缩关系特征。第三，提供了一个最终的框架，利用基于图像的视觉和基于图形的关系特征来完成药丸识别任务。在这个框架内，每个药丸的视觉表示被映射到图形嵌入空间，然后用于对图形表示执行注意力，从而产生有助于最终分类的语义丰富的上下文向量。据我们所知，这是第一个使用外部处方数据来建立药物之间的关联并使用这种辅助信息对其进行分类的研究。PIKA的架构是轻量级的，并且具有灵活性，可以集成到任何识别骨干中。实验结果表明，通过利用外部知识图谱，PIKA可以将识别准确率从4.8%提高到34.1%，与基线相比，F1得分。
英文摘要：
Identifying pills given their captured images under various conditions and backgrounds has been becoming more and more essential. Several efforts have been devoted to utilizing the deep learning-based approach to tackle the pill recognition problem in the literature. However, due to the high similarity between pills' appearance, misrecognition often occurs, leaving pill recognition a challenge. To this end, in this paper, we introduce a novel approach named PIKA that leverages external knowledge to enhance pill recognition accuracy. Specifically, we address a practical scenario (which we call contextual pill recognition), aiming to identify pills in a picture of a patient's pill intake. Firstly, we propose a novel method for modeling the implicit association between pills in the presence of an external data source, in this case, prescriptions. Secondly, we present a walk-based graph embedding model that transforms from the graph space to vector space and extracts condensed relational features of the pills. Thirdly, a final framework is provided that leverages both image-based visual and graph-based relational features to accomplish the pill identification task. Within this framework, the visual representation of each pill is mapped to the graph embedding space, which is then used to execute attention over the graph representation, resulting in a semantically-rich context vector that aids in the final classification. To our knowledge, this is the first study to use external prescription data to establish associations between medicines and to classify them using this aiding information. The architecture of PIKA is lightweight and has the flexibility to incorporate into any recognition backbones. The experimental results show that by leveraging the external knowledge graph, PIKA can improve the recognition accuracy from 4.8% to 34.1% in terms of F1-score, compared to baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
