arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2505.04643cs.CLcs.LG
收藏

高度不平衡文本数据中有限总体统计的预测驱动估计量:公共仇恨犯罪估计

Prediction-powered estimators for finite population statistics in highly imbalanced textual data: Public hate crime estimation

Department of Statistics, Uppsala University(统计系,乌普萨拉大学)

Hannes Waldetoft, Jakob Torgander, Måns Magnusson

AI总结:

本文提出将Transformer编码器预测作为辅助变量,与调查抽样估计量相结合,用于高度不平衡文本有限总体的参数估计,并在瑞典仇恨犯罪统计中验证了其高效性。

摘要:

在文本文档构成的有限总体中,当获取目标变量标签需要人工标注时,估计总体参数可能具有挑战性。为解决这一问题,我们将来自Transformer编码器神经网络的预测与成熟的调查抽样估计量相结合,将模型预测作为辅助变量。该方法在基于瑞典警方报告的瑞典仇恨犯罪统计中得到了应用验证。利用Hansen-Hurwitz估计量、差分估计和分层随机抽样估计,推导出每年仇恨犯罪数量及警方漏报情况的估计值。我们得出结论:如果有标注训练数据可用,所提出的方法能够提供非常高效的估计,同时减少人工标注所花费的时间。

英文摘要:

Estimating population parameters in finite populations of text documents can be challenging when obtaining the labels for the target variable requires manual annotation. To address this problem, we combine predictions from a transformer encoder neural network with well-established survey sampling estimators using the model predictions as an auxiliary variable. The applicability is demonstrated in Swedish hate crime statistics based on Swedish police reports. Estimates of the yearly number of hate crimes and the police's under-reporting are derived using the Hansen-Hurwitz estimator, difference estimation, and stratified random sampling estimation. We conclude that if labeled training data is available, the proposed method can provide very efficient estimates with reduced time spent on manual annotation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑