arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1801.07964cs.AIcs.HC
收藏

交互式机器学习系统的评价

Evaluation of Interactive Machine Learning Systems

INRA(法国国家农业科学研究院);Université Paris-Saclay(巴黎萨克雷大学);Univ Paris-Sud(巴黎第十一大学);CNRS(法国国家科学研究中心);LRI(计算机研究所);INRIA(法国国家信息与自动化研究所)

Nadia Boukhelifa, Anastasia Bezerianos, Evelyne Lutton

摘要:

交互式机器学习系统的评估仍然是一项艰巨的任务。这些系统向人类学习并适应人类,但与此同时,人类接受反馈并适应系统。要清楚地了解这些微妙的合作和共同适应机制是一项挑战。在本章中,我们报告了我们在设计和评估来自不同领域的各种交互式机器学习应用程序方面的经验。我们主张耦合两种类型的验证:以算法为中心的分析,研究系统的计算行为;以人为中心的评估,观察最终用户的应用程序的实用性和有效性。我们使用可视化分析应用程序进行引导搜索,使用交互式进化方法构建,作为我们工作的典范。我们的观察是,以人为本的设计和评估补充了算法分析,并在解决机器学习的“黑箱”效应方面发挥了重要作用。最后,我们讨论了需要人机交互方法的研究机会,以支持人类在交互式机器学习中扮演的可见和隐藏的角色。

英文摘要:

The evaluation of interactive machine learning systems remains a difficult task. These systems learn from and adapt to the human, but at the same time, the human receives feedback and adapts to the system. Getting a clear understanding of these subtle mechanisms of co-operation and co-adaptation is challenging. In this chapter, we report on our experience in designing and evaluating various interactive machine learning applications from different domains. We argue for coupling two types of validation: algorithm-centered analysis, to study the computational behaviour of the system; and human-centered evaluation, to observe the utility and effectiveness of the application for end-users. We use a visual analytics application for guided search, built using an interactive evolutionary approach, as an exemplar of our work. Our observation is that human-centered design and evaluation complement algorithmic analysis, and can play an important role in addressing the "black-box" effect of machine learning. Finally, we discuss research opportunities that require human-computer interaction methodologies, in order to support both the visible and hidden roles that humans play in interactive machine learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑