SynQP: A Framework and Metrics for Evaluating the Quality and Privacy Risk of Synthetic Data
Computer Science University of Waterloo(计算机科学大学水力学院);Public Health Sciences University of Waterloo(公共卫生科学大学水力学院);Public Health Agency of Canada Government of Canada(加拿大公共卫生机构政府)
Bing Hu, Yixin Li, Asma Bahamyirou, Helen Chen
备注:
7 Pages, 22nd Annual International Conference on Privacy, Security, and Trust (PST2025), Fredericton, Canada
发表信息:
2025 22nd Annual International Conference on Privacy, Security, and Trust (PST)
The use of synthetic data in health applications raises privacy concerns, yet the lack of open frameworks for privacy evaluations has slowed its adoption. A major challenge is the absence of accessible benchmark datasets for evaluating privacy risks, due to difficulties in acquiring sensitive data. To address this, we introduce SynQP, an open framework for benchmarking privacy in synthetic data generation (SDG) using simulated sensitive data, ensuring that original data remains confidential. We also highlight the need for privacy metrics that fairly account for the probabilistic nature of machine learning models. As a demonstration, we use SynQP to benchmark CTGAN and propose a new identity disclosure risk metric that offers a more accurate estimation of privacy risks compared to existing approaches. Our work provides a critical tool for improving the transparency and reliability of privacy evaluations, enabling safer use of synthetic data in health-related applications. % In our quality evaluations, non-private models achieved near-perfect machine-learning efficacy \(\ge0.97\). Our privacy assessments (Table II) reveal that DP consistently lowers both identity disclosure risk (SD-IDR) and membership-inference attack risk (SD-MIA), with all DP-augmented models staying below the 0.09 regulatory threshold. Code available at https://github.com/CAN-SYNH/SynQP
Evaluating the Potential of Federated Learning for Maize Leaf Disease Prediction
Institute of Exact and Technological Sciences (IEP)(精确与技术科学研究所);Federal University of Viçosa (UFV)(维索萨联邦大学);Faculty of Computing (FACOM)(计算机学院);Federal University of Uberlândia (UFU)(乌贝兰迪亚联邦大学)
Benchmarking Time Series Generation Methods for Privacy-Preserving Forecasting
University of Minho(米尼奥大学);Universidade do Porto(波尔图大学);Laboratory for Artificial Intelligence and Computer Science (LIACC)(人工智能与计算机科学实验室(LIACC));Fraunhofer Portugal AICOS(弗劳恩霍夫葡萄牙AICOS机构);INESCTEC;University of Coimbra(科英布拉大学)
Evaluating utility in synthetic banking microdata applications
Business School, University of Edinburgh(爱丁堡大学商学院);Central Bank of Paraguay(巴拉圭中央银行);Centre for Statistics, University of Edinburgh(爱丁堡大学统计中心);Centre for Financial Innovations, University of Edinburgh(爱丁堡大学金融创新中心)
Evaluating Selective Encryption Against Gradient Inversion Attacks
Information Networking Institute(信息网络研究所);Carnegie Mellon University(卡内基梅隆大学);TensorOpera AI;Department of Electrical and Computer Engineering(电气与计算机工程系)
Evaluating Precise Geolocation Inference Capabilities of Vision Language Models
Apart Research(阿帕特研究院);University of Maryland(马里兰大学);Hanoi University of Science and Technology(河内科技大学);University of Science and Technology of Hanoi(河内科学技术大学)
本文构建 Google Street View 全球分布基准,评估基础 VLM 及可调用工具的智能体在单图像地理定位上的能力,揭示其低误差表现及在线隐私风险。
Differentially Private Non Parametric Copulas: Generating synthetic data with non parametric copulas under privacy guarantees
School of Applied Sciences and Engineering, Universidad EAFIT(应用科学与工程学院,埃法特大学);Digital Health and Biomedical Technologies, Vicomtech Foundation, Basque Research and Technology Alliance (BRTA)(数字健康与生物医学技术,维科姆科技基金会,巴斯克研究与技术联盟(BRTA));eHealth Group, Biogipuzkoa Health Research Institute(eHealth组,比戈普西亚健康研究 institutes)