Synthetic data generation is a promising technique to facilitate the use of sensitive data while mitigating the risk of privacy breaches. However, for synthetic data to be useful in downstream analysis tasks, it needs to be of sufficient quality. Various methods have been proposed to measure the utility of synthetic data, but their results are often incomplete or even misleading. In this paper, we propose using density ratio estimation to improve quality evaluation for synthetic data, and thereby the quality of synthesized datasets. We show how this framework relates to and builds on existing measures, yielding global and local utility measures that are informative and easy to interpret. We develop an estimator which requires little to no manual tuning due to automatic selection of a nonparametric density ratio model. Through simulations, we find that density ratio estimation yields more accurate estimates of global utility than established procedures. A real-world data application demonstrates how the density ratio can guide refinements of synthesis models and can be used to improve downstream analyses. We conclude that density ratio estimation is a valuable tool in synthetic data generation workflows and provide these methods in the accessible open source R-package densityratio.
Evaluating quality in synthetic data generation for large tabular health datasets
Centre for Artificial Intelligence in Public Health Research, Robert Koch Institute(公共健康人工智能研究中心,德国罗伯特· Koch研究所);German Centre for Cancer Registry Data, Robert Koch Institute(癌症登记数据德国中心,德国罗伯特· Koch研究所);Medical Informatics, Berlin Institute of Health at Charité - Universitätsmedizin Berlin(医学信息学,柏林健康研究所(查理大学柏林医学中心))
TVineSynth: A Truncated C-Vine Copula Generator of Synthetic Tabular Data to Balance Privacy and Utility
Norwegian Computing Center(挪威计算中心);University of Oslo(奥斯陆大学);Technical University of Munich(慕尼黑工业大学);Munich Data Science Institute(慕尼黑数据科学研究所);Integreat - Norwegian Centre for Knowledge-driven Machine Learning(Integreat——挪威知识驱动型机器学习中心)
SMOTE-DP: Improving Privacy-Utility Tradeoff with Synthetic Data
University of Texas at Dallas Richardson, TX 75082(德克萨斯大学达拉斯分校);Vanderbilt University Medical Center Nashville, TN 37232(范德比尔特大学医学中心);Virginia Tech Blacksburg, VA 24061(弗吉尼亚理工大学)