arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.07860cs.CYcs.CL
收藏

Twitter文本摘要中的方言差异

Dialect Diversity in Text Summarization on Twitter

Yale University(耶鲁大学)

Vijay Keswani, L. Elisa Celis

摘要:

Twitter上的讨论涉及来自不同方言的不同社区的参与,通常需要将大量帖子汇总为具有代表性的样本以提供概要。然而,任何这样的代表性样本都应该充分描绘潜在的方言多样性,以呈现代表方言的不同参与社区的声音。提取摘要算法执行构建子集的任务,这些子集简洁地捕获任何给定帖子集合的主题。然而,我们观察到,有方言偏见的摘要产生的共同摘要方法,即,他们通常返回的摘要不足以代表某些方言。绝大多数现有的“公平”摘要方法需要社会显著属性标签(在这种情况下,方言),以确保所生成的摘要相对于社会显著属性是公平的。然而,在许多应用中,这些标签并不存在。此外,由于社交媒体中方言的不断发展,标记或准确推断每个社交媒体帖子的方言是不合理的。为了纠正方言偏见,我们采用了一个框架,将现有的文本摘要算法作为一个黑盒,并使用一小部分方言多样化的句子,返回一个相对更方言多样化的摘要。至关重要的是,这种方法不需要被总结的帖子具有方言标签,确保多样化过程独立于方言分类/识别模型。我们展示了我们的方法在Twitter数据集上的有效性,这些数据集包含由种族或性别定义的不同社会群体使用方言撰写的帖子;在所有情况下,与标准文本摘要方法相比,我们的方法都能提高方言多样性。

英文摘要:

Discussions on Twitter involve participation from different communities with different dialects and it is often necessary to summarize a large number of posts into a representative sample to provide a synopsis. Yet, any such representative sample should sufficiently portray the underlying dialect diversity to present the voices of different participating communities representing the dialects. Extractive summarization algorithms perform the task of constructing subsets that succinctly capture the topic of any given set of posts. However, we observe that there is dialect bias in the summaries generated by common summarization approaches, i.e., they often return summaries that under-represent certain dialects. The vast majority of existing "fair" summarization approaches require socially salient attribute labels (in this case, dialect) to ensure that the generated summary is fair with respect to the socially salient attribute. Nevertheless, in many applications, these labels do not exist. Furthermore, due to the ever-evolving nature of dialects in social media, it is unreasonable to label or accurately infer the dialect of every social media post. To correct for the dialect bias, we employ a framework that takes an existing text summarization algorithm as a blackbox and, using a small set of dialect-diverse sentences, returns a summary that is relatively more dialect-diverse. Crucially, this approach does not need the posts being summarized to have dialect labels, ensuring that the diversification process is independent of dialect classification/identification models. We show the efficacy of our approach on Twitter datasets containing posts written in dialects used by different social groups defined by race or gender; in all cases, our approach leads to improved dialect diversity compared to standard text summarization approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑