arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.02192cs.CYcs.CLcs.LG
收藏

利用大型语言模型自动化治理知识公地与情境完整性(GKC-CI)隐私政策标注

Automating Governing Knowledge Commons and Contextual Integrity (GKC-CI) Privacy Policy Annotations with Large Language Models

University of Chicago(芝加哥大学);Colgate University(科尔盖特大学)

Jake Chanenson, Madison Pickering, Noah Apthorpe

AI总结:

本文提出用大型语言模型自动完成隐私政策的GKC-CI参数标注,通过微调50个模型实现90.65%的专家级准确率,并规模化应用于456份政策。

摘要:

在隐私政策文本中识别情境完整性(CI)和治理知识公地(GKC)参数,可促进规范性隐私分析。然而,迄今为止,GKC-CI标注一直需要人工或众包工作。本文表明,使用大型语言模型可以自动执行高精度的隐私政策GKC-CI参数标注。我们基于来自16份隐私政策的21,588条真值GKC-CI标注,对50个开源和专有模型进行了微调。表现最佳的模型准确率达到90.65%,与专家在同一任务上的准确率相当。我们将表现最佳的模型应用于来自各类在线服务的456份隐私政策,证明了扩展GKC-CI标注用于隐私政策探索与分析的有效性。我们公开发布模型训练代码、训练和测试数据、一个标注可视化工具以及所有已标注政策,以供未来GKC-CI研究使用。

英文摘要:

Identifying contextual integrity (CI) and governing knowledge commons (GKC) parameters in privacy policy texts can facilitate normative privacy analysis. However, GKC-CI annotation has heretofore required manual or crowdsourced effort. This paper demonstrates that high-accuracy GKC-CI parameter annotation of privacy policies can be performed automatically using large language models. We fine-tune 50 open-source and proprietary models on 21,588 ground truth GKC-CI annotations from 16 privacy policies. Our best performing model has an accuracy of 90.65%, which is comparable to the accuracy of experts on the same task. We apply our best performing model to 456 privacy policies from a variety of online services, demonstrating the effectiveness of scaling GKC-CI annotation for privacy policy exploration and analysis. We publicly release our model training code, training and testing data, an annotation visualizer, and all annotated policies for future GKC-CI research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑