药物警戒AI系统的有效性和透明度:应用于不良事件医学文献监测的案例研究
Validation and Transparency in AI systems for pharmacovigilance: a case study applied to the medical literature monitoring of adverse events
摘要:
应用于生物医学文本的人工智能的最新进展为改善药物警戒活动提供了令人兴奋的机会,目前药物警戒活动受到不断增长的真实世界数据的影响。为了充分利用这些机会,应考虑现有的监管指导和行业最佳做法,以提高系统的整体可信度,并使其得到更广泛的采用。在本文中,我们提出了一个案例研究,探讨如何在药物警戒中实施经验证的AI系统的现有指南,重点关注科学文献中不良事件的医学文献监测(MLM)的具体任务。我们描述了一个人工智能系统,其设计目标是减少与主题专家密切合作内置的的MLM活动的工作量,并考虑为药物警戒和人工智能透明度方面的验证系统提供指导。特别是,我们利用公开披露作为有用的风险控制措施,以减少系统滥用并赢得用户信任。此外,我们目前的实验结果表明,该系统可以显着消除筛选工作,同时保持高水平的召回(过滤55%的不相关的文章平均,为0.99的目标召回可疑的不良文章),并提供了一个强大的方法来调整所需的召回,以适应特定的风险状况。
英文摘要:
Recent advances in artificial intelligence applied to biomedical text are opening exciting opportunities for improving pharmacovigilance activities currently burdened by the ever growing volumes of real world data. To fully realize these opportunities, existing regulatory guidance and industry best practices should be taken into consideration in order to increase the overall trustworthiness of the system and enable broader adoption. In this paper we present a case study on how to operationalize existing guidance for validated AI systems in pharmacovigilance focusing on the specific task of medical literature monitoring (MLM) of adverse events from the scientific literature. We describe an AI system designed with the goal of reducing effort in MLM activities built in close collaboration with subject matter experts and considering guidance for validated systems in pharmacovigilance and AI transparency. In particular we make use of public disclosures as a useful risk control measure to mitigate system misuse and earn user trust. In addition we present experimental results showing the system can significantly remove screening effort while maintaining high levels of recall (filtering 55% of irrelevant articles on average, for a target recall of 0.99 on suspected adverse articles) and provide a robust method for tuning the desired recall to suit a particular risk profile.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。