SNIP：一种改进的排序邻域方法用于家系数据去重
SNIP: An Adaptation of Sorted Neighborhood Methods for Deduplicating Pedigree Data
摘要：
系谱数据包含用于分析遗传性疾病的家族史信息。这些临床数据集可能包含重复记录，因为同一个系列多次访问诊所或临床医生出于测试目的输入多个版本的系列。从数据中得出的推论或使用它们进行训练或验证而不删除重复可能会导致无效的结论，因此识别重复是必不可少的。由于家族结构可能很复杂，因此无法直接应用现有的重复数据删除算法。我们首先通过检查谱系重复对家族风险预测模型的训练和验证的影响来激发重复数据删除的重要性。然后，我们引入了一个无监督的算法，我们称之为SNIP（排序邻域谱系），它建立在排序邻域方法，有效地找到和分类成对比较，利用固有的层次性质的谱系。我们进行了模拟研究，以评估该算法的性能，并找到参数配置的算法是能够准确地检测重复。然后，我们将该方法应用于来自风险服务的数据，其中包括超过30万个遗传性癌症高风险的家系，并发现了大量潜在的重复家庭。删除104，520个谱系（原始数据的33%）后，生成的Risk Service数据集现在可用于未来的分析、培训和验证。该算法可作为R包snipR在https://github.com/bayesmendel/snipR上获得。
英文摘要：
Pedigree data contain family history information that is used to analyze hereditary diseases. These clinical data sets may contain duplicate records due to the same family visiting a clinic multiple times or a clinician entering multiple versions of the family for testing purposes. Inferences drawn from the data or using them for training or validation without removing the duplicates could lead to invalid conclusions, and hence identifying the duplicates is essential. Since family structures can be complex, existing deduplication algorithms cannot be applied directly. We first motivate the importance of deduplication by examining the impact of pedigree duplicates on the training and validation of a familial risk prediction model. We then introduce an unsupervised algorithm, which we call SNIP (Sorted NeIghborhood for Pedigrees), that builds on the sorted neighborhood method to efficiently find and classify pairwise comparisons by leveraging the inherent hierarchical nature of the pedigrees. We conduct a simulation study to assess the performance of the algorithm and find parameter configurations where the algorithm is able to accurately detect the duplicates. We then apply the method to data from the Risk Service, which includes over 300,000 pedigrees at high risk of hereditary cancers, and uncover large clusters of potential duplicate families. After removing 104,520 pedigrees (33% of original data), the resulting Risk Service dataset can now be used for future analysis, training, and validation. The algorithm is available as an R package snipR available at https://github.com/bayesmendel/snipR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
