少可能性头脑风暴:使用语言模型生成替代假设
Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses
AI总结:
本文提出“少可能性头脑风暴”任务,并设计基于对比学习的受控文本生成方法,使模型能生成人类认为相关但不太可能的替代假设,在脑MRI解读和常识推理中优于现有方法。
摘要:
人类决策者从能够纠正其偏见的AI助手中获益最多。对于诸如根据检查发现生成放射学报告解读等问题,仅预测高度可能结果的系统可能用处较小,因为这些结果对用户而言已经显而易见。为了减轻人类决策中的偏见,值得考虑广泛的鉴别诊断,超越最可能的选项。我们引入了一项新任务——“少可能性头脑风暴”,该任务要求模型生成人类认为相关但不太可能发生的输出。我们在两种设置中探索该任务:脑MRI解读生成设置和日常常识推理设置。我们发现,以少可能性假设作为目标进行训练的基线方法生成的输出中,近一半被人类评价为要么是可能的、要么是不相关的;标准MLE训练并不有效。为了解决这一问题,我们提出了一种受控文本生成方法,该方法使用一种新颖的对比学习策略,鼓励模型根据人类判断区分生成可能输出与少可能输出。我们通过自动评估和人工评估将我们的方法与多个最先进的受控文本生成模型进行比较,并表明我们的模型生成少可能输出的能力得到了提升。
英文摘要:
A human decision-maker benefits the most from an AI assistant that corrects for their biases. For problems such as generating interpretation of a radiology report given findings, a system predicting only highly likely outcomes may be less useful, where such outcomes are already obvious to the user. To alleviate biases in human decision-making, it is worth considering a broad differential diagnosis, going beyond the most likely options. We introduce a new task, "less likely brainstorming," that asks a model to generate outputs that humans think are relevant but less likely to happen. We explore the task in two settings: a brain MRI interpretation generation setting and an everyday commonsense reasoning setting. We found that a baseline approach of training with less likely hypotheses as targets generates outputs that humans evaluate as either likely or irrelevant nearly half of the time; standard MLE training is not effective. To tackle this problem, we propose a controlled text generation method that uses a novel contrastive learning strategy to encourage models to differentiate between generating likely and less likely outputs according to humans. We compare our method with several state-of-the-art controlled text generation models via automatic and human evaluations and show that our models' capability of generating less likely outputs is improved.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。