LabelVizier：面向技术文本标注的交互式验证与重标注
LabelVizier: Interactive Validation and Relabeling for Technical Text Annotations
AI总结：
针对技术文本标注因需求变化易失效的问题，提出人在回路的工作流LabelVizier，结合领域知识实现交互式错误分析、标注验证与重标注，有效提升大规模多标签数据集的标注质量。
摘要：
随着数据驱动技术产生的文本数据快速积累，从非结构化原始文本中提取“数据标注”——即简洁、高质量的数据摘要——的任务变得日益重要。近期弱监督和众包技术的进展为大规模技术文本数据高效创建标注（标签）提供了有前景的解决方案。然而，由于标注需求、应用场景和建模目标的变化，此类标注在实践中可能会失败，需要领域专家进行标签验证和重标注。为解决此问题，我们提出了LabelVizier，这是一种人在回路的工作流，它结合领域知识和用户特定需求，揭示标注缺陷的可操作性见解，进而为大规模多标签数据集生成更高质量的标签。我们将该工作流实现为交互式笔记本，以促进灵活的错误分析、针对三种错误类型的深入标注验证，以及在不同数据规模上的高效标注重标注。我们通过两个用例和四位专家评审评估了该工作流的效率和通用性。结果表明，LabelVizier适用于各种应用场景，并协助具有不同知识背景的领域专家高效提升技术文本标注质量。
英文摘要：
With the rapid accumulation of text data produced by data-driven techniques, the task of extracting "data annotations"--concise, high-quality data summaries from unstructured raw text--has become increasingly important. The recent advances in weak supervision and crowd-sourcing techniques provide promising solutions to efficiently create annotations (labels) for large-scale technical text data. However, such annotations may fail in practice because of the change in annotation requirements, application scenarios, and modeling goals, where label validation and relabeling by domain experts are required. To approach this issue, we present LabelVizier, a human-in-the-loop workflow that incorporates domain knowledge and user-specific requirements to reveal actionable insights into annotation flaws, then produce better-quality labels for large-scale multi-label datasets. We implement our workflow as an interactive notebook to facilitate flexible error profiling, in-depth annotation validation for three error types, and efficient annotation relabeling on different data scales. We evaluated the efficiency and generalizability of our workflow with two use cases and four expert reviews. The results indicate that LabelVizier is applicable in various application scenarios and assist domain experts with different knowledge backgrounds to efficiently improve technical text annotation quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
