arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.06905cs.LG
收藏

将专家反馈转化为模型更新的观点

Perspectives on Incorporating Expert Feedback into Model Updates

Carnegie Mellon University(卡内基梅隆大学);University of Cambridge(剑桥大学);The Alan Turing Institute(阿兰·图灵研究所)

Valerie Chen, Umang Bhatt, Hoda Heidari, Adrian Weller, Ameet Talwalkar

摘要:

机器学习(ML)从业者越来越多地承担着开发与非技术专家的价值观和目标相一致的模型的任务。然而,对于从业者如何将领域专业知识转化为ML更新的考虑还不够。在本文中,我们考虑如何系统地捕捉从业者和专家之间的互动。我们设计了一个分类,以匹配专家的反馈类型与从业者的更新。从业者可以在观察或域级别接收来自专家的反馈,并将该反馈转换为对数据集、损失函数或参数空间的更新。我们回顾了ML和人机交互的现有工作,以描述这种反馈更新分类法,并强调对纳入非技术专家反馈的考虑不足。最后,我们提出了一系列开放性问题,这些问题是从我们提出的分类法和随后的调查中自然产生的。

英文摘要:

Machine learning (ML) practitioners are increasingly tasked with developing models that are aligned with non-technical experts' values and goals. However, there has been insufficient consideration on how practitioners should translate domain expertise into ML updates. In this paper, we consider how to capture interactions between practitioners and experts systematically. We devise a taxonomy to match expert feedback types with practitioner updates. A practitioner may receive feedback from an expert at the observation- or domain-level, and convert this feedback into updates to the dataset, loss function, or parameter space. We review existing work from ML and human-computer interaction to describe this feedback-update taxonomy, and highlight the insufficient consideration given to incorporating feedback from non-technical experts. We end with a set of open questions that naturally arise from our proposed taxonomy and subsequent survey.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑