网络对话中讽刺语的可靠注释
Getting Reliable Annotations for Sarcasm in Online Dialogues
摘要：
在线论坛中使用的语言在许多方面不同于传统的语言资源，如新闻。一个区别是讽刺等非字面、主观对话行为的使用和频率。无论目标是发展对话中的讽刺理论，还是设计可靠检测讽刺的自动方法，一个主要的挑战就是很难获得足够多的可靠标记的例子。在本文中，我们描述了我们的工作方法实现高度可靠的讽刺注释从未经训练的注释机土耳其。我们探讨了一些常见的统计可靠性措施，如卡帕，Karger的，多数类，EM。我们表明，与简单的测量（例如假设正确的标签是大多数土耳其人应用的标签）相比，更复杂的测量似乎并不能为我们的数据产生更好的结果。
英文摘要：
The language used in online forums differs in many ways from that of traditional language resources such as news. One difference is the use and frequency of nonliteral, subjective dialogue acts such as sarcasm. Whether the aim is to develop a theory of sarcasm in dialogue, or engineer automatic methods for reliably detecting sarcasm, a major challenge is simply the difficulty of getting enough reliably labelled examples. In this paper we describe our work on methods for achieving highly reliable sarcasm annotations from untrained annotators on Mechanical Turk. We explore the use of a number of common statistical reliability measures, such as Kappa, Karger's, Majority Class, and EM. We show that more sophisticated measures do not appear to yield better results for our data than simple measures such as assuming that the correct label is the one that a majority of Turkers apply.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
