数学辅导中的效用保持去标识化：MathEd-PII基准数据集中的数值歧义研究
Utility-Preserving De-Identification for Math Tutoring: Investigating Numeric Ambiguity in the MathEd-PII Benchmark Dataset
AI总结：
针对数学辅导对话中数值表达式与标识符相似导致过度去标识化的问题，提出MathEd-PII基准数据集，并采用领域感知提示策略（F1达0.821）在保持数据效用的同时有效检测PII。
摘要：
大规模共享对话数据是推进教学科学的关键，但严格的去标识化仍是一大障碍。在数学辅导记录中，数值表达式常与结构化标识符（如日期或ID）相似，导致通用个人身份信息（PII）检测系统过度编辑核心教学内容，降低数据效用。本研究探讨如何在保持教育效用的同时检测PII，重点关注这一“数值歧义”问题。我们引入了MathEd-PII，这是首个用于数学辅导对话中PII检测的基准数据集，通过人机协同的LLM标注构建。利用基于密度的分割，我们发现虚假PII编辑集中在数学密集区域，证实数值歧义是主要失败模式。随后比较了四种检测策略：Presidio基线以及三种基于LLM的方法（基础提示、数学感知提示和片段感知提示）。领域感知提示（包括数学感知F1: 0.802和片段感知F1: 0.821）显著优于基线（F1: 0.379），同时减少了数值假阳性，表明去标识化必须融入领域上下文以保持分析效用。本研究提供了新的基准和证据，表明辅导数据的效用保持去标识化需要领域感知建模。
英文摘要：
Large-scale sharing of dialogue data is key to advancing the science of teaching and learning, yet rigorous de-identification remains a major barrier. In mathematics tutoring transcripts, numeric expressions frequently resemble structured identifiers (e.g., dates or IDs), leading generic Personally Identifiable Information (PII) detection systems to over-redact core instructional content and reduce data utility. This work asks how to detect PII while preserving educational utility, focusing on this "numeric ambiguity" problem. We introduce MathEd-PII, the first benchmark dataset for PII detection in math tutoring dialogues, built with human-in-the-loop LLM annotation. Using density-based segmentation, we show that false PII redactions cluster in math-dense regions, confirming numeric ambiguity as a key failure mode. We then compare four detection strategies: a Presidio baseline and three LLM-based approaches with basic, math-aware, and segment-aware prompting. Domain-aware prompting, including both math-aware (F1: 0.802) and segment-aware versions (F1: 0.821), substantially outperforms the baseline (F1: 0.379) while reducing numeric false positives, demonstrating that de-identification must incorporate domain context to preserve analytic utility. This work provides a new benchmark and evidence that utility-preserving de-identification for tutoring data requires domain-aware modeling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
