arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.18601stat.MLcs.LG
收藏

在线决策调解

Online Decision Mediation

University of Cambridge(剑桥大学);UCLA(加州大学洛杉矶分校);Alan Turing Institute(艾伦·图灵研究所)

Daniel Jarrett, Alihan Hüyük, Mihaela van der Schaar

AI总结:

本文提出在线决策调解框架,在人类决策与专家反馈之间学习一个中介策略,通过弃权反馈权衡即时损失与未来泛化改进,并在多个数据集上验证其优于现有基准。

摘要:

考虑学习一个决策支持助手,作为(神谕式)专家行为与(不完美的)人类行为之间的中介:在每个时刻,算法观察到一个易犯错的智能体所选择的动作,并决定是*接受*该智能体的决策、用替代方案*干预*,还是*请求*专家的意见。例如,在临床诊断中,完全自主的机器行为往往超出伦理允许范围,因此现实世界中的决策支持通常仅限于监测和预测。相反,这样的中介将在前者(纯粹规定性的)和后者(纯粹描述性的)方法之间取得审慎的平衡,同时为人类错误与专家反馈之间提供高效的接口。在这项工作中,我们首先形式化*在线决策调解*这一序列问题——即利用*弃权(不执行)反馈*从零开始同时学习和评估调解策略:在每一轮中,遵从神谕可以避免出错风险,但会产生前期代价,并揭示原本隐藏的专家动作作为新的训练数据点。其次,我们提出并论证了一种解决方案,该方案寻求在(即时)损失项与(未来)泛化误差改进之间进行权衡;在此过程中,我们指出了传统老虎机算法可能失败的原因。最后,通过在多种数据集上的实验和敏感性分析,我们展示了在调解策略、所学模型以及整个决策系统方面的性能指标上,相对于适用基准的一致提升。

英文摘要:

Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior: At each time, the algorithm observes an action chosen by a fallible agent, and decides whether to *accept* that agent's decision, *intervene* with an alternative, or *request* the expert's opinion. For instance, in clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances, thus real-world decision support is often limited to monitoring and forecasting. Instead, such an intermediary would strike a prudent balance between the former (purely prescriptive) and latter (purely descriptive) approaches, while providing an efficient interface between human mistakes and expert feedback. In this work, we first formalize the sequential problem of *online decision mediation* -- that is, of simultaneously learning and evaluating mediator policies from scratch with *abstentive feedback*: In each round, deferring to the oracle obviates the risk of error, but incurs an upfront penalty, and reveals the otherwise hidden expert action as a new training data point. Second, we motivate and propose a solution that seeks to trade off (immediate) loss terms against (future) improvements in generalization error; in doing so, we identify why conventional bandit algorithms may fail. Finally, through experiments and sensitivities on a variety of datasets, we illustrate consistent gains over applicable benchmarks on performance measures with respect to the mediator policy, the learned model, and the decision-making system as a whole.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑