PIIBench：一个统一的多源基准语料库用于个人可识别信息检测
PIIBench: A Unified Multi-Source Benchmark Corpus for Personally Identifiable Information Detection
AI总结：
PIIBench整合多个数据集，构建统一的PII检测基准，通过标准化标签和分层划分提升评估难度，展示现有系统在PII检测上的不足。
摘要：
我们提出了PIIBench，一个统一的基准语料库，用于自然语言文本中的个人可识别信息（PII）检测。现有PII检测资源分散在领域特定语料库中，标注方案互不兼容，阻碍了检测系统的系统性比较。我们整合了十个公开可用的数据集，涵盖合成PII语料库、多语言命名实体识别（NER）基准和金融领域标注文本，得到包含2369883个标注序列和335万实体提及的语料库，覆盖48种标准PII实体类型。我们开发了一个原则性的标准化流程，将80多种来源特定的标签变体映射到标准化的BIO标注方案，应用基于频率的近似缺失实体类型的抑制，并产生分层的80/10/10训练/验证/测试划分，保持来源分布。为建立基准难度，我们评估了八个已发布的系统，涵盖基于规则的引擎（Microsoft Presidio）、通用目的NER模型（spaCy、BERT-base NER、XLM-RoBERTa NER、SpanMarker mBERT、SpanMarker BERT）、专门针对PII的模型（Piiranha DeBERTa）和金融NER专家（XtremeDistil FiNER）。所有系统在跨度级别F1得分低于0.14，最佳系统（Presidio，F1=0.1385）在大多数实体类型上仍产生零召回。这些结果直接量化了领域隔离问题，展示了PIIBench比任何现有单一来源PII数据集更具挑战性和更全面的评估难度。数据集构建流程和基准评估代码在https://github.com/pritesh-2711/pii-bench公开。
英文摘要：
We present PIIBench, a unified benchmark corpus for Personally Identifiable Information (PII) detection in natural language text. Existing resources for PII detection are fragmented across domain-specific corpora with mutually incompatible annotation schemes, preventing systematic comparison of detection systems. We consolidate ten publicly available datasets spanning synthetic PII corpora, multilingual Named Entity Recognition (NER) benchmarks, and financial domain annotated text, yielding a corpus of 2,369,883 annotated sequences and 3.35 million entity mentions across 48 canonical PII entity types. We develop a principled normalization pipeline that maps 80+ source-specific label variants to a standardized BIO tagging scheme, applies frequency-based suppression of near absent entity types, and produces stratified 80/10/10 train/validation/test splits preserving source distribution. To establish baseline difficulty, we evaluate eight published systems spanning rule-based engines (Microsoft Presidio), general purpose NER models (spaCy, BERT-base NER, XLM-RoBERTa NER, SpanMarker mBERT, SpanMarker BERT), a PII-specific model (Piiranha DeBERTa), and a financial NER specialist (XtremeDistil FiNER). All systems achieve span-level F1 below 0.14, with the best system (Presidio, F1=0.1385) still producing zero recall on most entity types. These results directly quantify the domain-silo problem and demonstrate that PIIBench presents a substantially harder and more comprehensive evaluation challenge than any existing single source PII dataset. The dataset construction pipeline and benchmark evaluation code are publicly available at https://github.com/pritesh-2711/pii-bench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
