arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.01791cs.CL
收藏

行善还是做对?探索常识因果推理模型的弱点

Doing Good or Doing Right? Exploring the Weakness of Commonsense Causal Reasoning Models

Shanghai University of Finance and Economics(上海财经大学)

Mingyue Han, Yinglin Wang

AI总结:

本文揭示了PLM在COPA任务上的语义相似性偏差问题,通过添加正则化损失有效缓解了该偏差,提升了模型的泛化能力和在无偏数据集BCOPA-CE上的鲁棒性。

摘要:

预训练语言模型(PLM)在Choice of Plausible Alternatives(COPA)任务上取得了惊人的性能。然而,PLM是否真正获得了因果推理能力仍是一个问题。在本文中,我们研究了语义相似性偏差问题,并通过特定攻击揭示了当前COPA模型的脆弱性。先前解决未平衡词元分布的表面线索的方案仍遇到相同的语义偏差问题,甚至由于使用了更多训练数据而更加严重。我们通过简单添加正则化损失来缓解此问题,实验结果表明,该方案不仅提高了模型的泛化能力,还帮助模型在具有无偏词元分布且更难区分因果关系的挑战性数据集BCOPA-CE上表现得更鲁棒。

英文摘要:

Pretrained language models (PLM) achieve surprising performance on the Choice of Plausible Alternatives (COPA) task. However, whether PLMs have truly acquired the ability of causal reasoning remains a question. In this paper, we investigate the problem of semantic similarity bias and reveal the vulnerability of current COPA models by certain attacks. Previous solutions that tackle the superficial cues of unbalanced token distribution still encounter the same problem of semantic bias, even more seriously due to the utilization of more training data. We mitigate this problem by simply adding a regularization loss and experimental results show that this solution not only improves the model's generalization ability, but also assists the models to perform more robustly on a challenging dataset, BCOPA-CE, which has unbiased token distribution and is more difficult for models to distinguish cause and effect.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑