轻量语言模型在PII遮蔽中的比较研究及其在真实对话文本中的部署
A Comparative Study of Light-weight Language Models for PII Masking and their Deployment for Real Conversational Texts
AI总结：
本文比较了轻量级语言模型在PII遮蔽中的性能，发现Mistral在F1和召回率上表现更优，但T5在实时对话中更高效且可控。
摘要：
自动屏蔽个人可识别信息（PII）对于保护隐私的对话系统至关重要。尽管当前前沿大语言模型在PII屏蔽方面表现出色，但数据处理和计算成本方面的担忧促使探索轻量级模型是否能实现同等性能。我们通过在AI4Privacy基准测试集上微调T5-small和Mistral-Instruct-v0.3，比较了编码器-解码器和解码器-only架构。我们创建了不同的数据集变体以研究标签标准化和PII表示，涵盖了24种标准化的PII类别和更高粒度的设置。使用实体级和字符级指标、类型准确率和精确匹配进行评估，结果显示两种轻量模型在PII屏蔽任务中的性能与前沿LLM相当。标签规范化在所有架构中一致提高了性能。Mistral在F1和召回率上更高，具有更强的鲁棒性，但生成延迟显著更高。T5虽然在对话文本中稳健性较低，但提供了更可控的结构化输出和更低的推理成本，促使其用于实时Discord机器人中的真实PII遮蔽。在实时消息上的评估显示，在非正式输入下性能会下降。这些结果澄清了准确率、鲁棒性和计算效率之间的权衡，证明轻量模型可以在解决前沿LLM相关数据处理问题的同时提供有效的PII屏蔽。
英文摘要：
Automated masking of Personally Identifiable Information (PII) is critical for privacy-preserving conversational systems. While current frontier large language models demonstrate strong PII masking capabilities, concerns about data handling and computational costs motivate exploration of whether lightweight models can achieve comparable performance. We compare encoder-decoder and decoder-only architectures by fine-tuning T5-small and Mistral-Instruct-v0.3 on English datasets constructed from the AI4Privacy benchmark. We create different dataset variants to study label standardization and PII representation, covering 24 standardized PII categories and higher-granularity settings. Evaluation using entity-level and character-level metrics, type accuracy, and exact match shows that both lightweight models achieve performance comparable to frontier LLMs for PII masking tasks. Label normalization consistently improves performance across architectures. Mistral achieves higher F1 and recall with greater robustness across PII types but incurs significantly higher generation latency. T5, while less robust in conversational text, offers more controllable structured outputs and lower inference cost, motivating its use in a real-time Discord bot for real-world PII redaction. Evaluation on live messages reveals performance degradation under informal inputs. These results clarify trade-offs between accuracy, robustness, and computational efficiency, demonstrating that lightweight models can provide effective PII masking while addressing data handling concerns associated with frontier LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
