arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.09913cs.CL
收藏

临床序列标注中标注的粒度和成本分析

Analyzing the Granularity and Cost of Annotation in Clinical Sequence Labeling

Duke University(杜克大学);The Australian National University(澳大利亚国立大学)

Haozhan Sun, Chenchen Xu, Hanna Suominen

摘要:

正如最近的顶级研究所示,在监督式机器学习(ML)中,注释良好的数据集对研究人员来说比以往任何时候都更加重要。然而,数据集注释过程及其相关的人力成本仍然被忽视。在这项工作中,我们分析了序列标记中的注释粒度和ML性能之间的关系,使用护理交接班的临床记录。我们首先研究了一个模型,仅从文本语言特征,没有额外的信息的基础上护理知识。我们发现,在此粒度下,该序列标签器在大多数类别中表现良好。然后,我们进一步包括护士的额外手动注释,并发现序列标记性能几乎保持不变。最后,我们给社区提供了一个指导和参考,因为投资回报率低,没有必要甚至不建议进行详细的粒度注释。因此,我们建议强调其他功能,如文本知识,为研究人员和从业者作为一个具有成本效益的来源,以提高序列标记性能。

英文摘要:

Well-annotated datasets, as shown in recent top studies, are becoming more important for researchers than ever before in supervised machine learning (ML). However, the dataset annotation process and its related human labor costs remain overlooked. In this work, we analyze the relationship between the annotation granularity and ML performance in sequence labeling, using clinical records from nursing shift-change handover. We first study a model derived from textual language features alone, without additional information based on nursing knowledge. We find that this sequence tagger performs well in most categories under this granularity. Then, we further include the additional manual annotations by a nurse, and find the sequence tagging performance remaining nearly the same. Finally, we give a guideline and reference to the community arguing it is not necessary and even not recommended to annotate in detailed granularity because of a low Return on Investment. Therefore we recommend emphasizing other features, like textual knowledge, for researchers and practitioners as a cost-effective source for increasing the sequence labeling performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑