MLaaS中的私有训练集检查
Private Training Set Inspection in MLaaS
AI总结:
本文首次针对MLaaS中客户无法访问私有训练集的问题,提出结合成员推断的影子训练和多示例学习特征化方案,估计训练集的多样性与公平性指标,在文本分类实验中实现0.87准确率和99.3%置信度。
摘要:
机器学习即服务(MLaaS)是一种流行的基于云的解决方案,面向希望使用机器学习模型但缺乏训练数据、计算资源或机器学习专业知识的客户。在这种情况下,训练数据集通常是机器学习或数据公司的私有财产,客户无法访问,但客户仍需要一种方法来确认训练数据集符合他们的期望并满足公平性等监管措施。然而,目前没有现有工作解决上述客户的担忧。这项工作是解决该问题的首次尝试,以数据起源作为切入点。我们首先定义了起源成员度量,并在此基础上定义了多样性和公平性指标,以解决客户的担忧。然后,我们提出了一种策略来估计这两个指标在不可访问的训练数据集中的值,该策略结合了来自成员推断的影子训练技术和多示例学习中的高效特征化方案。评估包含基于语言BERT模型的文本评论极性分类应用。实验结果表明,我们的解决方案在成员检查中可以实现高达0.87的准确率,在检查多样性和公平性分布方面可以达到高达99.3%的置信度。
英文摘要:
Machine Learning as a Service (MLaaS) is a popular cloud-based solution for customers who aim to use an ML model but lack training data, computation resources, or expertise in ML. In this case, the training datasets are typically a private possession of the ML or data companies and are inaccessible to the customers, but the customers still need an approach to confirm that the training datasets meet their expectations and fulfil regulatory measures like fairness. However, no existing work addresses the above customers' concerns. This work is the first attempt to solve this problem, taking data origin as an entry point. We first define origin membership measurement and based on this, we then define diversity and fairness metrics to address customers' concerns. We then propose a strategy to estimate the values of these two metrics in the inaccessible training dataset, combining shadow training techniques from membership inference and an efficient featurization scheme in multiple instance learning. The evaluation contains an application of text review polarity classification applications based on the language BERT model. Experimental results show that our solution can achieve up to 0.87 accuracy for membership inspection and up to 99.3% confidence in inspecting diversity and fairness distribution.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。