如何计数咳嗽:一种用于评估自动咳嗽检测算法性能的基于事件的框架
How to Count Coughs: An Event-Based Framework for Evaluating Automatic Cough Detection Algorithm Performance
AI总结:
针对现有咳嗽检测算法评估指标与临床需求不匹配、易受类别不平衡影响的问题,本研究提出基于事件的评估指标,搭建开源评估框架,并给出相关实践指南,为评估算法的临床相关性能提供基础。
摘要:
慢性咳嗽疾病十分普遍,且评估难度大,因为这类疾病的评估依赖于患者填写的关于咳嗽频率的主观问卷。搭载机器学习(ML)算法的可穿戴设备有望实现日常咳嗽的量化,为临床医生提供客观指标来追踪症状、评估治疗效果。然而,目前咳嗽计数算法的前沿评估指标与临床医生关注的相关信息之间存在不匹配。大多数研究聚焦于区分咳嗽样本与非咳嗽样本,这无法直接提供具有临床相关性的结果,比如咳嗽事件的数量或其时间模式。此外,特异性、准确率等常用指标可能会因类别不平衡而产生偏差。我们提出采用符合临床指南中重要咳嗽计数终点要求的基于事件的评估指标。我们使用一个ML分类器来说明传统的基于样本的准确率测量方法的缺陷,强调其会因数据集类别不平衡和样本窗口长度而产生波动。我们还提出了一个开源的基于事件的评估框架,用于测试算法识别咳嗽事件、排除误报的性能。我们提供了基于事件的咳嗽计数的相关示例和最佳实践指南,这是评估算法临床相关性能的必要第一步。
英文摘要:
Chronic cough disorders are widespread and challenging to assess because they rely on subjective patient questionnaires about cough frequency. Wearable devices running Machine Learning (ML) algorithms are promising for quantifying daily coughs, providing clinicians with objective metrics to track symptoms and evaluate treatments. However, there is a mismatch between state-of-the-art metrics for cough counting algorithms and the information relevant to clinicians. Most works focus on distinguishing cough from non-cough samples, which does not directly provide clinically relevant outcomes such as the number of cough events or their temporal patterns. In addition, typical metrics such as specificity and accuracy can be biased by class imbalance. We propose using event-based evaluation metrics aligned with clinical guidelines on significant cough counting endpoints. We use an ML classifier to illustrate the shortcomings of traditional sample-based accuracy measurements, highlighting their variance due to dataset class imbalance and sample window length. We also present an open-source event-based evaluation framework to test algorithm performance in identifying cough events and rejecting false positives. We provide examples and best practice guidelines in event-based cough counting as a necessary first step to assess algorithm performance with clinical relevance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。