arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1810.09648cs.AI
收藏

AI能为我做什么:在合作游戏中评估机器学习解释

What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play

University of Maryland(马里兰大学)

Shi Feng, Jordan Boyd-Graber

摘要:

机器学习是一种重要的决策工具,但其道德和负责任的应用需要对其可解释性和实用性进行严格审查:这是一个未充分研究的问题,特别是对于自然语言处理模型。我们提出了一个评价的解释与真正的人类用户,解释的有效性是衡量多少,它提高了人类的表现在一个真正的任务。我们设计了一个接地,现实的人机合作设置使用问答任务,Quizbowl。我们招募了知识专家和新手来玩这个游戏,电脑作为他们的队友,通过三种不同的解释来传达它的预测。我们还为自然语言处理人在回路设置提供设计指导。

英文摘要:

Machine learning is an important tool for decision making, but its ethical and responsible application requires rigorous vetting of its interpretability and utility: an understudied problem, particularly for natural language processing models. We propose an evaluation of interpretation on a real task with real human users, where the effectiveness of interpretation is measured by how much it improves human performance. We design a grounded, realistic human-computer cooperative setting using a question answering task, Quizbowl. We recruit both trivia experts and novices to play this game with computer as their teammate, who communicates its prediction via three different interpretations. We also provide design guidance for natural language processing human-in-the-loop settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑