arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.13610cs.CLcs.AI
收藏

让你的决策令人信服!一个统一的两阶段框架:自归因与决策制定

Make Your Decision Convincing! A Unified Two-Stage Framework: Self-Attribution and Decision-Making

Harbin Institute of Technology(哈尔滨工业大学)

Yanrui Du, Sendong Zhao, Haochun Wang, Yuhan Chen, Rui Bai, Zewen Qiang, Muzhen Cai, Bing Qin

AI总结:

针对模型决策与生成理由之间联系不可靠的问题,提出了统一的两阶段框架SADM,在ERASER基准的五个数据集上验证了其在任务性能、理由质量及建立可靠联系方面的有效性,并探索了半监督场景下的潜力。

摘要:

用自然语言解释黑盒模型行为在各种NLP任务中取得了令人印象深刻的结果。最近的研究探索了将输入文本的子序列作为理由,为用户提供支持模型决策的证据。尽管现有框架在生成高质量理由的同时实现了高任务性能,但它们忽略了对生成理由与模型决策之间不可靠联系的考量。简而言之,模型可能在归因错误理由的同时做出正确决策,或者在归因正确理由的同时做出糟糕决策。为了缓解这一问题,我们提出了一个统一的两阶段框架,称为自归因与决策制定(SADM)。通过对来自ERASER基准的五个推理数据集进行广泛实验,我们证明我们的框架不仅在生成理由与模型决策之间建立了更可靠的联系,而且在任务性能和理由质量上取得了有竞争力的结果。此外,我们还探索了该框架在半监督场景下的潜力。

英文摘要:

Explaining black-box model behavior with natural language has achieved impressive results in various NLP tasks. Recent research has explored the utilization of subsequences from the input text as a rationale, providing users with evidence to support the model decision. Although existing frameworks excel in generating high-quality rationales while achieving high task performance, they neglect to account for the unreliable link between the generated rationale and model decision. In simpler terms, a model may make correct decisions while attributing wrong rationales, or make poor decisions while attributing correct rationales. To mitigate this issue, we propose a unified two-stage framework known as Self-Attribution and Decision-Making (SADM). Through extensive experiments on five reasoning datasets from the ERASER benchmark, we demonstrate that our framework not only establishes a more reliable link between the generated rationale and model decision but also achieves competitive results in task performance and the quality of rationale. Furthermore, we explore the potential of our framework in semi-supervised scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑