笑头：Transformers能检测出是什么让句子有趣吗？
Laughing Heads: Can Transformers Detect What Makes a Sentence Funny?
摘要：
幽默的自动检测对自然语言处理提出了巨大的挑战。基于transformer的系统最近在这项任务上取得了显着的成果，但它们通常（1）~在严肃与幽默文本来自完全不同来源的设置中进行评估，（2）~专注于基准性能，而不提供对模型如何工作的见解。我们在这两个方面都取得了进展，通过训练和分析基于transformer的幽默识别模型，最近推出的数据集由最小对对齐的句子组成，一个严重，另一个幽默。我们发现，尽管我们的对齐数据集比以前的数据集要难得多，但基于transformer的模型以高准确率（78%）识别对齐对中的幽默句子。在仔细的错误分析中，我们描述了简单与困难的实例。最后，通过分析注意力权重，我们获得了重要的见解的机制，Transformers识别幽默。最值得注意的是，我们发现了明确的证据，表明一个单一的注意头学会识别使测试句子幽默的单词，即使在训练时没有获得这些信息。
英文摘要：
The automatic detection of humor poses a grand challenge for natural language processing. Transformer-based systems have recently achieved remarkable results on this task, but they usually (1)~were evaluated in setups where serious vs humorous texts came from entirely different sources, and (2)~focused on benchmarking performance without providing insights into how the models work. We make progress in both respects by training and analyzing transformer-based humor recognition models on a recently introduced dataset consisting of minimal pairs of aligned sentences, one serious, the other humorous. We find that, although our aligned dataset is much harder than previous datasets, transformer-based models recognize the humorous sentence in an aligned pair with high accuracy (78%). In a careful error analysis, we characterize easy vs hard instances. Finally, by analyzing attention weights, we obtain important insights into the mechanisms by which transformers recognize humor. Most remarkably, we find clear evidence that one single attention head learns to recognize the words that make a test sentence humorous, even without access to this information at training time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
