个性化聊天机器人可信度评级
Personalized Chatbot Trustworthiness Ratings
摘要：
会话代理，通常被称为聊天机器人，越来越多地部署在许多领域，以允许人们在试图解决特定问题时进行自然交互。鉴于其广泛使用，重要的是为其用户提供方法和工具，以提高用户对聊天机器人的各种属性的认识，包括用户可能认为重要的非功能属性，以便信任特定的聊天机器人。例如，用户可能希望使用没有偏见的聊天机器人，不使用辱骂性语言，不向其他用户泄露信息，并且以适合用户认知水平的风格进行响应。在本文中，我们解决的设置聊天机器人不能被修改，其训练数据不能被访问，但一个中立的一方希望评估和沟通其可信度的用户，定制用户的优先级超过各种信任问题。这样的评级可以帮助用户在其他聊天机器人中进行选择，开发人员测试他们的系统，商业领袖为他们的产品定价，监管机构制定政策。我们设想了一种聊天机器人的个性化评级方法，该方法依赖于每个问题的单独评级模块，以及用户在相关信任问题中检测到的优先级排序，以生成聊天机器人可信度的聚合个性化评级。该方法是独立的特定的信任问题，是参数的聚合过程，从而允许无缝的泛化。我们说明了它的一般用途，将其与实时聊天机器人集成，并在四个对话数据集和代表性的用户简档上进行评估，并通过用户调查进行验证。
英文摘要：
Conversation agents, commonly referred to as chatbots, are increasingly deployed in many domains to allow people to have a natural interaction while trying to solve a specific problem. Given their widespread use, it is important to provide their users with methods and tools to increase users awareness of various properties of the chatbots, including non-functional properties that users may consider important in order to trust a specific chatbot. For example, users may want to use chatbots that are not biased, that do not use abusive language, that do not leak information to other users, and that respond in a style which is appropriate for the user's cognitive level. In this paper, we address the setting where a chatbot cannot be modified, its training data cannot be accessed, and yet a neutral party wants to assess and communicate its trustworthiness to a user, tailored to the user's priorities over the various trust issues. Such a rating can help users choose among alternative chatbots, developers test their systems, business leaders price their offering, and regulators set policies. We envision a personalized rating methodology for chatbots that relies on separate rating modules for each issue, and users' detected priority orderings among the relevant trust issues, to generate an aggregate personalized rating for the trustworthiness of a chatbot. The method is independent of the specific trust issues and is parametric to the aggregation procedure, thereby allowing for seamless generalization. We illustrate its general use, integrate it with a live chatbot, and evaluate it on four dialog datasets and representative user profiles, validated with user surveys.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
