arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2602.02766cs.LGcs.CLcs.CR
收藏

私有微调的LLM在表格数据中保留时间动态

Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Data

NYU(纽约大学);Columbia University(哥伦比亚大学);Google Research(谷歌研究)

Lucas Rosenblatt, Peihan Liu, Ryan McKenna, Natalia Ponomareva

AI总结:

本文提出PATH框架,利用私有微调的LLM保留表格数据的时间动态,有效捕捉长距离依赖并提升合成数据质量。

摘要:

关于差分隐私合成表格数据的研究大多集中在独立同分布的行上,每条记录对应一个唯一个体。这种观点忽视了纵向数据集中的时间复杂性,如电子健康记录,其中用户贡献整个(子)表的连续事件。虽然从业者可能尝试通过将用户历史扁平化为高维向量来使用标准边缘机制来建模此类数据,但我们证明这种策略是不足的。即使在保持有效边缘分布的情况下,扁平化也无法保留时间一致性。我们引入PATH,一种新的生成框架,将完整的表格视为合成的单位,并利用私有微调的大型语言模型的自回归能力。广泛的评估显示,PATH有效捕捉了传统方法所遗漏的长距离依赖性。实证上,我们的方法将分布距离减少到真实轨迹的幅度超过60%,并且与领先的边缘机制相比,将状态转换误差减少了近50%,同时保持相似的边缘保真度。

英文摘要:

Research on differentially private synthetic tabular data has largely focused on independent and identically distributed rows where each record corresponds to a unique individual. This perspective neglects the temporal complexity in longitudinal datasets, such as electronic health records, where a user contributes an entire (sub) table of sequential events. While practitioners might attempt to model such data by flattening user histories into high-dimensional vectors for use with standard marginal-based mechanisms, we demonstrate that this strategy is insufficient. Flattening fails to preserve temporal coherence even when it maintains valid marginal distributions. We introduce PATH, a novel generative framework that treats the full table as the unit of synthesis and leverages the autoregressive capabilities of privately fine-tuned large language models. Extensive evaluations show that PATH effectively captures long-range dependencies that traditional methods miss. Empirically, our method reduces the distributional distance to real trajectories by over 60% and reduces state transition errors by nearly 50% compared to leading marginal mechanisms while achieving similar marginal fidelity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑