当机器学习变得个性化：评估预测与解释
When Machine Learning Gets Personal: Evaluating Prediction and Explanation
AI总结：
本文探讨了个性化模型对预测和解释的影响，提出统一框架量化个性化对两者的影响，并通过实证分析揭示了数据集统计对测试效果的限制。
摘要：
在高风险领域如医疗健康中，用户往往期望分享个人信息给机器学习系统会带来实际利益，如更准确的诊断和更清晰的解释。然而，这一假设的有效性尚无充分研究。我们提出一个统一框架，量化个性化如何影响预测和解释。我们显示，其对预测和解释的影响可能不同：模型在预测不变的情况下，可能变得更或更不具解释性。对于实际应用，我们研究了检测个性化对人口群体影响的标准假设检验。我们推导出其概率误差的有限样本下界，作为群体大小、个人属性数量和期望个性化收益的函数。这提供了行动见解，如哪些数据集特征是测试效果所必需的，或给定数据集能测试的最大效果。我们应用我们的框架到现实世界表格数据集，利用特征归因方法，揭示了某些情况下由于数据集统计无法测试效果的场景。我们的结果强调了在个性化模型中联合评估预测和解释的必要性，以及设计模型和数据集以提供足够信息进行此类评估的重要性。
英文摘要：
In high-stakes domains like healthcare, users often expect that sharing personal information with machine learning systems will yield tangible benefits, such as more accurate diagnoses and clearer explanations of contributing factors. However, the validity of this assumption remains largely unexplored. We propose a unified framework to quantify how personalizing a model influences both prediction and explanation. We show that its impacts on prediction and explanation can diverge: a model may become more or less explainable even when prediction is unchanged. For practical settings, we study a standard hypothesis test for detecting personalization effects on demographic groups. We derive a finite-sample lower bound on its probability of error as a function of group sizes, number of personal attributes, and desired benefit from personalization. This provides actionable insights, such as which dataset characteristics are necessary to test an effect, or the maximum effect that can be tested given a dataset. We apply our framework to real-world tabular datasets using feature-attribution methods, uncovering scenarios where effects are fundamentally untestable due to the dataset statistics. Our results highlight the need for joint evaluation of prediction and explanation in personalized models and the importance of designing models and datasets with sufficient information for such evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
