arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.12786cs.CL
收藏

从格赖斯量的角度看“我不知道”问题

Measuring the `I don't know' Problem through the Lens of Gricean Quantity

Johns Hopkins University(约翰霍普金斯大学);New York University(纽约大学)

Huda Khayrallah, João Sedoc

摘要:

我们考虑通过Grice的对话准则(1975)的镜头神经生成对话模型的内在评价。基于数量(信息量)的准则,我们提出了相对话语数量(RUQ)来诊断“我不知道”问题,其中对话系统产生通用的响应。语言动机的RUQ诊断比较的模型分数的通用响应的参考响应。我们发现,对于合理的基线模型,“我不知道”是首选的大部分时间的参考,但这可以减少到小于5%的超参数调整。RUQ允许直接分析“我不知道”的问题,这已经解决,但没有分析以前的工作。

英文摘要:

We consider the intrinsic evaluation of neural generative dialog models through the lens of Grice's Maxims of Conversation (1975). Based on the maxim of Quantity (be informative), we propose Relative Utterance Quantity (RUQ) to diagnose the `I don't know' problem, in which a dialog system produces generic responses. The linguistically motivated RUQ diagnostic compares the model score of a generic response to that of the reference response. We find that for reasonable baseline models, `I don't know' is preferred over the reference the majority of the time, but this can be reduced to less than 5% with hyperparameter tuning. RUQ allows for the direct analysis of the `I don't know' problem, which has been addressed but not analyzed by prior work.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑