Calpric：利用众包与主动学习对隐私政策进行包容性细粒度标注
Calpric: Inclusive and Fine-grain Labeling of Privacy Policies with Crowdsourcing and Active Learning
AI总结：
针对隐私政策训练数据获取成本高的问题，提出Calpric，结合自动文本选择、主动学习与众包标注，低成本生成大规模平衡数据集，实现细粒度标注。
摘要：
在隐私政策上训练准确的深度学习模型的一个重大挑战是获取大量且全面的训练数据的成本和难度。为解决这些挑战，我们提出了Calpric，它结合了自动文本选择与分割、主动学习以及众包标注者的使用，以低成本为隐私政策生成一个大规模、平衡的训练集。自动文本选择与分割简化了标注任务，使得来自众包平台（如Amazon的Mechanical Turk）的未经训练的标注者能够与受过训练的标注者（如法学院学生）相媲美，同时也降低了标注者间的一致性，从而减少了标注成本。拥有可靠的训练标签使得能够使用主动学习，该技术使用更少的训练样本来高效覆盖输入空间，进一步降低成本并改善数据集中的类别和数据类别平衡。这些技术的结合使Calpric能够生成在更广泛的数据类别上准确的模型，并提供比以往工作更详细、更细粒度的标签。我们的众包流程使Calpric能够以每个标注文本段约0.92-1.71美元的成本获得可靠的标注数据。Calpric的训练过程还生成了一个包含16K个隐私政策文本段、跨越9个数据类别、正负样本平衡的标注数据集。
英文摘要：
A significant challenge to training accurate deep learning models on privacy policies is the cost and difficulty of obtaining a large and comprehensive set of training data. To address these challenges, we present Calpric , which combines automatic text selection and segmentation, active learning and the use of crowdsourced annotators to generate a large, balanced training set for privacy policies at low cost. Automated text selection and segmentation simplifies the labeling task, enabling untrained annotators from crowdsourcing platforms, like Amazon's Mechanical Turk, to be competitive with trained annotators, such as law students, and also reduces inter-annotator agreement, which decreases labeling cost. Having reliable labels for training enables the use of active learning, which uses fewer training samples to efficiently cover the input space, further reducing cost and improving class and data category balance in the data set. The combination of these techniques allows Calpric to produce models that are accurate over a wider range of data categories, and provide more detailed, fine-grain labels than previous work. Our crowdsourcing process enables Calpric to attain reliable labeled data at a cost of roughly $0.92-$1.71 per labeled text segment. Calpric 's training process also generates a labeled data set of 16K privacy policy text segments across 9 Data categories with balanced positive and negative samples.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
