利用社会语言学知识训练和评估隐私保护情感识别模型的仿人模型
Human-Imitating Metrics for Training and Evaluating Privacy Preserving Emotion Recognition Models Using Sociolinguistic Knowledge
摘要：
隐私保护是任何实际应用程序的关键组成部分。但是，在依赖于机器学习后端的应用程序中，隐私是具有挑战性的，因为模型通常会捕获比模型最初训练的更多的内容，从而导致敏感信息的潜在泄漏。在本文中，我们提出了一个自动和可量化的度量，使我们能够评估人类的感知模型的能力，以保护隐私方面的敏感变量。在本文中，我们专注于基于显着性的解释，解释突出输入文本的区域，推断黑盒模型的内部工作。我们使用一般与隐私保护模型的解释差异与社会语言学偏见相关的程度来告知度量设计。我们展示了某些寻求保护隐私的常用方法如何与人类对隐私保护的看法不一致，从而导致对模型声明的不信任。我们证明了我们提出的度量的多功能性，通过验证其实用程序来测量跨语料库泛化隐私和情感。最后，我们进行众包实验，以评估评估者的倾向，以选择一个特定的模型为给定的目的时，提供模型的解释，并显示出积极的关系与建议的度量。据我们所知，我们迈出了第一步，提出了自动和可量化的指标，这些指标最符合人类对模型隐私保护能力的感知，从而实现了具有成本效益的模型开发。
英文摘要：
Privacy preservation is a crucial component of any real-world application. But, in applications relying on machine learning backends, privacy is challenging because models often capture more than what the model was initially trained for, resulting in the potential leakage of sensitive information. In this paper, we propose an automatic and quantifiable metric that allows us to evaluate humans' perception of a model's ability to preserve privacy with respect to sensitive variables. In this paper, we focus on saliency-based explanations, explanations that highlight regions of the input text, to infer internal workings of a black box model. We use the degree with which differences in interpretation of general vs privacy preserving models correlate with sociolinguistic biases to inform metric design. We show how certain commonly-used methods that seek to preserve privacy do not align with human perception of privacy preservation leading to distrust about model's claims. We demonstrate the versatility of our proposed metric by validating its utility for measuring cross corpus generalization for both privacy and emotion. Finally, we conduct crowdsourcing experiments to evaluate the inclination of the evaluators to choose a particular model for a given purpose when model explanations are provided, and show a positive relationship with the proposed metric. To the best of our knowledge, we take the first step in proposing automatic and quantifiable metrics that best align with human perception of model's ability for privacy preservation, allowing for cost-effective model development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
