通过突出潜在错误并建议修正,以不完美指令成功引导人类
Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Corrections
AI总结:
本研究开发HEAR系统,通过向用户预警指令潜在错误并给出修正建议的不确定性传达方式,在长时序住宅模拟引导任务中提升了人类决策成功率与位置准确性,验证了该方法的实际价值。
摘要:
语言模型在不熟悉的场景中不可避免会出错。但通过有效传达不确定性,它们仍能在这些情境下引导人类做出合理决策。我们通过开发HEAR(人类错误感知引导系统)验证了这一思路,该系统能在模拟住宅环境中成功引导人类,即便其生成的指令可能不准确。与仅向用户提供自身生成指令的系统不同,HEAR会就指令中的潜在错误向用户发出警告,并给出修正建议。这类丰富的不确定性信息有效避免了误导,缩小了用户的搜索范围。针对80名用户的评估显示,与仅向用户展示指令的方式相比,HEAR的任务成功率提升了13%,最终位置误差距离降低了29%。有趣的是,我们发现通过为用户提供探索可能性,HEAR能激励他们在任务中做出更多尝试,最终带来更高的成功率。据我们所知,这项工作首次证明了不确定性传达在长时序序列决策问题中的实际效用。
英文摘要:
Language models will inevitably err in situations with which they are unfamiliar. However, by effectively communicating uncertainties, they can still guide humans toward making sound decisions in those contexts. We demonstrate this idea by developing HEAR, a system that can successfully guide humans in simulated residential environments despite generating potentially inaccurate instructions. Diverging from systems that provide users with only the instructions they generate, HEAR warns users of potential errors in its instructions and suggests corrections. This rich uncertainty information effectively prevents misguidance and reduces the search space for users. Evaluation with 80 users shows that HEAR achieves a 13% increase in success rate and a 29% reduction in final location error distance compared to only presenting instructions to users. Interestingly, we find that offering users possibilities to explore, HEAR motivates them to make more attempts at the task, ultimately leading to a higher success rate. To our best knowledge, this work is the first to show the practical benefits of uncertainty communication in a long-horizon sequential decision-making problem.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。