利用机器学习辅助抽样和大语言模型标注测量违规内容的普及率
Measuring the Prevalence of Policy Violating Content with ML Assisted Sampling and LLM Labeling
AI总结：
本文提出了一种基于机器学习和大语言模型的系统，用于高效测量违反政策内容的普及率，通过概率抽样和多模态标注提升测量准确性。
摘要：
内容安全团队需要反映用户实际体验的指标，而不仅仅是报告的内容。我们研究普及率：在某一天内，用户观看（印象）中违反给定政策的内容所占的比例。准确测量普及率具有挑战性，因为违规内容往往罕见，且人工标注成本高，导致频繁、具有代表性的研究变得缓慢。我们提出了一种基于设计的测量系统，该系统（i）从印象流中每日抽取概率样本，使用机器学习辅助权重来集中标注预算在高曝光和高风险内容上，同时保持无偏性；（ii）使用受政策提示和黄金集验证的多模态大语言模型标注抽样内容；（iii）生成一致的普及率估计值，包括置信区间和仪表盘钻取功能。关键设计目标是单一全局样本支持多个维度：同一天的样本通过后分层估计支持普及率、观众地理、内容年龄和其他细分领域。我们描述了统计估计器、方差和置信区间构造、标签质量监控以及使系统在不同政策间可配置的工程工作流程。
英文摘要：
Content safety teams need metrics that reflect what users actually experience, not only what is reported. We study prevalence: the fraction of user views (impressions) that went to content violating a given policy on a given day. Accurate prevalence measurement is challenging because violations are often rare and human labeling is costly, making frequent, platform-representative studies slow. We present a design-based measurement system that (i) draws daily probability samples from the impression stream using ML-assisted weights to concentrate label budget on high-exposure and high-risk content while preserving unbiasedness, (ii) labels sampled items with a multimodal LLM governed by policy prompts and gold-set validation, and (iii) produces design-consistent prevalence estimates with confidence intervals and dashboard drilldowns. A key design goal is one global sample with many pivots: the same daily sample supports prevalence by surface, viewer geography, content age, and other segments through post-stratified estimation. We describe the statistical estimators, variance and confidence interval construction, label-quality monitoring, and an engineering workflow that makes the system configurable across policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
