AI 中文总结
针对实体解析领域假设变量存在唯一不变真值的缺陷，提出适配多重真值的指数族分布混合贝叶斯模型，经模拟与意大利家庭财富调查验证，可改善数据整合与下游推断。
AI 中文摘要
在从政府到生态学的众多应用中，整合来自不同且含噪声的数据源的数据对下游推断至关重要。然而，可能不存在唯一标识符来清晰链接同一实体的记录。实体解析（也称为去重或记录链接）会合并此类数据库以识别重复项，从而为推断提供更完整的数据。近年来，统计学和计算机科学中的大量方法假设链接所用的每个变量都存在唯一且不变的真值。我们认为这一限制性假设在实践中常被违反，导致研究人员丢弃有用数据并使下游推断产生偏差。例如，受访者的教育状况可能在两次不同调查间确实发生了变化，但现有方法会将至少一个观测值视为真值的失真版本，即便两者均正确。本文中，我们提出一种新颖的实体解析模型，通过引入指数族分布混合模型来全面适配“多重真值”，该模型还可处理多种数据类型。我们提供马尔可夫链蒙特卡洛（MCMC）程序和变分推断两种拟合选项，以适配大规模数据集。我们通过模拟实验和意大利家庭财富纵向调查的链接工作证明了该方法的价值。
英文摘要
In many applications, from government to ecology, integrating data from diverse and noisy sources is critical for downstream inference. However, a unique identifier to link records cleanly from the same entity may not exist. Entity resolution (also referred to as de-duplication or record linkage) merges such databases to identify duplicates, allowing for more complete data for inference. A multitude of methods from statistics and computer science in recent years assume a single, immutable true value for each variable used in linkage. We argue this restrictive assumption is often violated in practice, leading researchers to discard useful data and biasing downstream inference. For example, a respondent's education status may truly change between two different surveys, yet existing methods would treat at least one observation as a distorted version of the truth, even though both are correct. In this paper, we propose a novel entity resolution model that comprehensively accommodates "multiple truths" by introducing a mixture of exponential family distributions that further handles multiple data types. We provide options to fit the model with Markov chain Monte Carlo routines and variational inference for massive datasets. We demonstrate the method's value via simulation and linking a longitudinal survey of Italian household wealth.
Comments74 pages
