基于代理表示的大规模大语言模型应用的隐私保护数据漂移检测与恢复
Privacy-Preserving Data Drift Detection and Recovery for Large-Scale LLM Applications via Proxy Representations
浏览论文内容
中文总结 AI 辅助
针对大规模LLM应用的隐私约束导致的评估与漂移追踪难题，提出ProxyDrift框架，基于非PII代理表示实现无敏感数据暴露的漂移监测与合成数据生成，实验验证其对齐度达RA~0.9。
中文摘要 AI 辅助
大规模部署的大语言模型（LLM）应用面临一个根本挑战：隐私约束禁止直接检查用户交互，导致难以获取任何代表性评估数据集，也难以追踪生产流量的持续演变。我们提出ProxyDrift框架，该框架可在不访问原始用户数据的情况下，（i）识别并测量生产流量与离线评估集之间的漂移，（ii）相应地构建和刷新这些评估集。我们的方法完全基于非个人可识别信息（non-PII）代理表示运行：即从基于LLM的用户交互分类中得出的结构化多维描述符。我们引入了：（1）经概率校准、考虑冗余的（RA）对齐分数，该分数通过互信息聚合各维度的漂移测量值；（2）条件采样器，可生成符合维度间依赖关系的合成代理；（3）往返一致性分析，用于揭示生成器/分类器的分歧并指导代理分类体系的优化；（4）反馈关联分析，将各维度和各取值的代理分布与用户满意度关联，从而发现可操作的失败和成功模式。ProxyDrift服务于数亿用户，可实现连续漂移监测和针对性合成数据生成，同时不暴露敏感用户数据。实验证实其具备强往返一致性、合成查询与人类查询在判别器层面的不可区分性，以及与生产环境的紧密端到端对齐（RA~0.9）。
英文摘要
LLM applications deployed at scale face a fundamental challenge: privacy constraints prevent direct inspection of user interactions, making it difficult to obtain any representative evaluation dataset or to track the ongoing evolution of production traffic. We present ProxyDrift, a framework that (i) identifies and measures drift between production traffic and offline evaluation sets, and (ii) constructs and refreshes those evaluation sets accordingly; all without access to raw user data. Our approach operates entirely on non-PII proxy representations: structured, multi-dimensional descriptors derived from LLM-based classification of user interactions. We introduce (1) a chance-calibrated, redundancy-aware (RA) alignment score that aggregates per-dimension drift measurements via mutual information; (2) a conditional sampler that generates synthetic proxies respecting inter-dimensional dependencies; (3) a roundtrip consistency analysis that exposes generator/classifier disagreements and guides proxy taxonomy refinement; and (4) a feedback-linkage analysis that ties per-dimension and per-value proxy distributions to user satisfaction, surfacing actionable failure and success modes. Serving hundreds of millions of users, ProxyDrift enables continuous drift monitoring and targeted synthetic data generation without exposing sensitive user data. Experiments confirm strong roundtrip consistency, discriminator-level indistinguishability of synthetic queries from human queries, and tight end-to-end alignment (RA~0.9) with production.
发表机构
- Microsoft Corporation（微软公司）
机构由 AI 辅助整理，请以论文原文为准。
