arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1912.12413stat.MLcs.LG
收藏

作为医疗器械的基于机器学习的软件修改的批准政策:生物蠕变研究

Approval policies for modifications to Machine Learning-Based Software as a Medical Device: A study of bio-creep

University of Washington(华盛顿大学)

Jean Feng, Scott Emerson, Noah Simon

摘要:

在医疗保健领域成功部署机器学习算法需要对其性能和安全性进行仔细评估。迄今为止,FDA在上市前批准锁定算法,并要求未来的更新进行单独的上市前审查。然而,这否定了机器学习的一个关键特性--从不断增长的数据集中学习并随着时间的推移而改进的能力。本文框架设计的审批政策,我们称之为自动算法变更协议(aACP),作为一个在线假设检验问题。由于这一过程与新药的非劣效性测试有明显的相似性,我们研究了重复测试和采用修改可能导致预测准确性逐渐恶化,在药物开发文献中也称为“生物筛选”。我们考虑简单的政策,人们可能会考虑,但不一定提供任何错误率保证,以及政策,提供错误率控制。对于后者,我们定义了两个适合这种情况的在线错误率:Bad Approval Count(BAC)和Bad Approval and Benchmark Ratios(BABR)。我们使用aACP-BAC和aACP-BABR策略在恒定人口和数据源的简单设置中控制这些速率,该策略结合了α投资,组顺序和守门方法。在模拟研究中,当使用没有错误率保证的策略时,生物蠕变经常发生,而aACP-BAC和-BABR控制生物蠕变的速率,而不会实质上影响我们批准有益修改的能力。

英文摘要:

Successful deployment of machine learning algorithms in healthcare requires careful assessments of their performance and safety. To date, the FDA approves locked algorithms prior to marketing and requires future updates to undergo separate premarket reviews. However, this negates a key feature of machine learning--the ability to learn from a growing dataset and improve over time. This paper frames the design of an approval policy, which we refer to as an automatic algorithmic change protocol (aACP), as an online hypothesis testing problem. As this process has obvious analogy with noninferiority testing of new drugs, we investigate how repeated testing and adoption of modifications might lead to gradual deterioration in prediction accuracy, also known as ``biocreep'' in the drug development literature. We consider simple policies that one might consider but do not necessarily offer any error-rate guarantees, as well as policies that do provide error-rate control. For the latter, we define two online error-rates appropriate for this context: Bad Approval Count (BAC) and Bad Approval and Benchmark Ratios (BABR). We control these rates in the simple setting of a constant population and data source using policies aACP-BAC and aACP-BABR, which combine alpha-investing, group-sequential, and gate-keeping methods. In simulation studies, bio-creep regularly occurred when using policies with no error-rate guarantees, whereas aACP-BAC and -BABR controlled the rate of bio-creep without substantially impacting our ability to approve beneficial modifications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑