In this work, we propose to use linguistic annotations as a basis for a \textit{Discourse-Aware Semantic Self-Attention} encoder that we employ for reading comprehension on long narrative texts. We extract relations between discourse units, events and their arguments as well as coreferring mentions, using available annotation tools. Our empirical evaluation shows that the investigated structures improve the overall performance, especially intra-sentential and cross-sentential discourse relations, sentence-internal semantic role relations, and long-distance coreference relations. We show that dedicating self-attention heads to intra-sentential relations and relations connecting neighboring sentences is beneficial for finding answers to questions in longer contexts. Our findings encourage the use of discourse-semantic annotations to enhance the generalization capacity of self-attention models for reading comprehension.
In this paper, we propose the scheme for annotating large-scale multi-party chat dialogues for discourse parsing and machine comprehension. The main goal of this project is to help understand multi-party dialogues. Our dataset is based on the Ubuntu Chat Corpus. For each multi-party dialogue, we annotate the discourse structure and question-answer pairs for dialogues. As we know, this is the first large scale corpus for multi-party dialogues discourse parsing, and we firstly propose the task for multi-party dialogues machine reading comprehension.
Harbin Institute of Technology（哈尔滨工业大学）；National University of Singapore（新加坡国立大学）
Multiparty Dialogue Machine Reading Comprehension (MRC) differs from traditional MRC as models must handle the complex dialogue discourse structure, previously unconsidered in traditional MRC. To fully exploit such discourse structure in multiparty dialogue, we present a discourse-aware dialogue graph neural network, DADgraph, which explicitly constructs the dialogue graph using discourse dependency links and discourse relations. To validate our model, we perform experiments on the Molweni corpus, a large-scale MRC dataset built over multiparty dialogue annotated with discourse structure. Expe…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Colorado Boulder（科罗拉多大学博尔德分校）；University of California San Diego（加州大学圣地亚哥分校）
While there has been substantial progress in text comprehension through simple factoid question answering, more holistic comprehension of a discourse still presents a major challenge (Dunietz et al., 2020). Someone critically reflecting on a text as they read it will pose curiosity-driven, often open-ended questions, which reflect deep understanding of the content and require complex reasoning to answer (Ko et al., 2020; Westera et al., 2020). A key challenge in building and evaluating models for this type of discourse comprehension is the lack of annotated data, especially since collecting an…
University of Chinese Academy of Sciences（中国科学院大学）；Tencent Inc.（腾讯公司）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
Current event-centric knowledge graphs highly rely on explicit connectives to mine relations between events. Unfortunately, due to the sparsity of connectives, these methods severely undermine the coverage of EventKGs. The lack of high-quality labelled corpora further exacerbates that problem. In this paper, we propose a knowledge projection paradigm for event relation extraction: projecting discourse knowledge to narratives by exploiting the commonalities between them. Specifically, we propose Multi-tier Knowledge Projection Network (MKPNet), which can leverage multi-tier discourse knowledge …
UNC Chapel Hill（北卡罗来纳大学教堂山分校）；Tencent AI Lab（腾讯人工智能实验室）
Comprehending a dialogue requires a model to capture diverse kinds of key information in the utterances, which are either scattered around or implicitly implied in different turns of conversations. Therefore, dialogue comprehension requires diverse capabilities such as paraphrasing, summarizing, and commonsense reasoning. Towards the objective of pre-training a zero-shot dialogue comprehension model, we develop a novel narrative-guided pre-training strategy that learns by narrating the key information from a dialogue input. However, the dialogue-narrative parallel corpus for such a pre-trainin…
Harbin Institute of Technology（哈尔滨工业大学）；National University of Singapore（新加坡国立大学）；Peng Cheng Laboratory（鹏城实验室）
Research into the area of multiparty dialog has grown considerably over recent years. We present the Molweni dataset, a machine reading comprehension (MRC) dataset with discourse structure built over multiparty dialog. Molweni's source samples from the Ubuntu Chat Corpus, including 10,000 dialogs comprising 88,303 utterances. We annotate 30,066 questions on this corpus, including both answerable and unanswerable questions. Molweni also uniquely contributes discourse dependency annotations in a modified Segmented Discourse Representation Theory (SDRT; Asher et al., 2016) style for all of its mu…
Multi-party dialogue machine reading comprehension (MRC) raises an even more challenging understanding goal on dialogue with more than two involved speakers, compared with the traditional plain passage style MRC. To accurately perform the question-answering (QA) task according to such multi-party dialogue, models have to handle fundamentally different discourse relationships from common non-dialogue plain text, where discourse relations are supposed to connect two far apart utterances in a linguistics-motivated way.To further explore the role of such unusual discourse structure on the correlat…
Booz Allen Hamilton（博思艾伦咨询公司）；International Water Management Institute（国际水管理研究院）；NOVA Information Management School (NOVA IMS)（NOVA信息管理学院）；New York University Abu Dhabi（纽约大学阿布扎比分校）
People often turn to social media to comment upon and share information about major global events. Accordingly, social media is receiving increasing attention as a rich data source for understanding people's social, political and economic experiences of extreme weather events. In this paper, we contribute two novel methodologies that leverage Twitter discourse to characterize narratives and identify unmet needs in response to Cyclone Amphan, which affected 18 million people in May 2020.
University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
Human understanding of narrative texts requires making commonsense inferences beyond what is stated explicitly in the text. A recent model, COMET, can generate such implicit commonsense inferences along several dimensions such as pre- and post-conditions, motivations, and mental states of the participants. However, COMET was trained on commonsense inferences of short phrases, and is therefore discourse-agnostic. When presented with each sentence of a multi-sentence narrative, it might generate inferences that are inconsistent with the rest of the narrative. We present the task of discourse-awa…
University of California Irvine（加州大学欧文分校）；IBM Research（IBM研究院）；WeChat AI, Tencent（腾讯微信人工智能）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；University of Washington（华盛顿大学）；University of Notre Dame（圣母大学）；Syracuse University（雪城大学）；IBM Research Ireland（IBM爱尔兰研究院）；Hong Kong Univ. of Sci and Tech（香港科技大学）；Georgia Institute of Technology（佐治亚理工学院）；University of California Los Angeles（加州大学洛杉矶分校）；Columbia University（哥伦比亚大学）
Question answering (QA) is a fundamental means to facilitate assessment and training of narrative comprehension skills for both machines and young children, yet there is scarcity of high-quality QA datasets carefully designed to serve this purpose. In particular, existing datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. Drawing on the reading education research, we introduce FairytaleQA, a dataset focusing on narrative comprehension of kindergarten to eighth-grade students. Generated by educational experts based on an evidence-ba…
Detecting Narrative Shifts through Persistent Structures: A Topological Analysis of Media Discourse
Department of Cyber Intelligence and Data Science（网络智能与数据科学系）；Biological and Computational Intelligence Center（生物与计算智能中心）；National Intelligence University（国家智能大学）
Multi-party multi-turn dialogue comprehension brings unprecedented challenges on handling the complicated scenarios from multiple speakers and criss-crossed discourse relationship among speaker-aware utterances. Most existing methods deal with dialogue contexts as plain texts and pay insufficient attention to the crucial speaker-aware clues. In this work, we propose an enhanced speaker-aware model with masking attention and heterogeneous graph networks to comprehensively capture discourse clues from both sides of speaker property and speaker-aware relationships. With such comprehensive speaker…
Discourse vs emissions: Analysis of corporate narratives, symbolic practices, and mimicry through LLMs
QUANT AI Lab（QUANT AI 实验室）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；Centre d’Économie de la Sorbonne (CES), Maison des Sciences Économiques (MSE), Université Paris 1 Panthéon-Sorbonne（索邦大学经济研究中心（CES）、经济科学之家（MSE））；Institut Louis Bachelier, Palais Brongniart, 28 Place de la Bourse, 75002 Paris, France（路易·巴希尔研究所）；EDC Paris Business School（巴黎商学院）
What distinguishes conspiracy from critical narratives? A computational analysis of oppositional discourse
Universitat Politècnica de València（瓦伦西亚理工大学）；Ruđer Bošković Institute（鲁杰尔·博什科维奇研究所）；Symanto Research（Symanto研究公司）；Universitat de València（瓦伦西亚大学）；Universitat de Barcelona（巴塞罗那大学）；ValgrAI - Valencian Graduate School and Research Network of Artificial Intelligence（瓦伦西亚人工智能研究生学院与研究网络）
Interinstitutional Center for Computational Linguistics (NILC)（跨机构计算语言学中心）；Institute of Mathematical and Computer Sciences（数学与计算机科学研究所）；University of São Paulo（圣保罗大学）
Automated discourse analysis tools based on Natural Language Processing (NLP) aiming at the diagnosis of language-impairing dementias generally extract several textual metrics of narrative transcripts. However, the absence of sentence boundary segmentation in the transcripts prevents the direct application of NLP methods which rely on these marks to function properly, such as taggers and parsers. We present the first steps taken towards automatic neuropsychological evaluation based on narrative discourse analysis, presenting a new automatic sentence segmentation method for impaired speech. Our…
Reading comprehension (RC)---in contrast to information retrieval---requires integrating information and reasoning about events, entities, and their relations across a full document. Question answering is conventionally used to assess RC ability, in both artificial agents and children learning to read. However, existing RC datasets and tasks are dominated by questions that can be solved by selecting answers using superficial information (e.g., local context similarity or global term frequency); they thus fail to test for the essential integrative aspect of RC. To encourage progress on deeper c…
This paper tackles the problem of reading comprehension over long narratives where documents easily span over thousands of tokens. We propose a curriculum learning (CL) based Pointer-Generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty. This can be interpreted as a form of domain randomization and/or generative pretraining during training. To this end, the usage of the Pointer-Generator softens the requirement of having the answer within the context, enabling us to construct diverse…
Many tasks aim to measure machine reading comprehension (MRC), often focusing on question types presumed to be difficult. Rarely, however, do task designers start by considering what systems should in fact comprehend. In this paper we make two key contributions. First, we argue that existing approaches do not adequately define comprehension; they are too unsystematic about what content is tested. Second, we present a detailed definition of comprehension -- a "Template of Understanding" -- for a widely useful class of texts, namely short narratives. We then conduct an experiment that strongly s…
Extracting narrative signals from public discourse: a network-based approach
Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学科学研究所）；Laboratoire Lattice, École Normale Supérieure - PSL - CNRS - Univ. Sorbonne Nouvelle（格勒诺布尔大学拉提实验室）；médialab, Sciences Po（社会科学高等学院媒体实验室）；Brussels School of Governance, Vrije Universiteit Brussel（布鲁塞尔治理学校，自由大学布鲁塞尔）；imec-SMIT, Vrije Universiteit Brussel（imec-SMIT，自由大学布鲁塞尔）
Nanjing University of Posts and Telecommunications（南京邮电大学）；Guangzhou University（广州大学）
ChatGPT is a publicly available chatbot that can quickly generate texts on given topics, but it is unknown whether the chatbot is really superior to human writers in all aspects of writing and whether its writing quality can be prominently improved on the basis of updating commands. Consequently, this study compared the writing performance on a narrative topic by ChatGPT and Chinese intermediate English (CIE) learners so as to reveal the chatbot's advantage and disadvantage in writing. The data were analyzed in terms of five discourse components using Coh-Metrix (a special instrument for analy…
Multi-Level Narrative Evaluation Outperforms Lexical Features for Mental Health
Institute for Artificial Intelligence（人工智能研究院）；School of Psychological and Cognitive Sciences（心理学与认知科学学院）；School of Intelligence Science and Technology（智能科学与技术学院）；State Key Laboratory of General AI（通用人工智能国家重点实验室）；Beijing Key Laboratory of Behavior and Mental Health（北京行为与心理健康重点实验室）；PKU-Changsha Institute for Computing and Digital Economy（北京大学长沙计算与数字经济研究院）
Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）；Elsevier Research Data Services（爱思唯尔研究数据服务）
We propose a deep learning model for identifying structure within experiment narratives in scientific literature. We take a sequence labeling approach to this problem, and label clauses within experiment narratives to identify the different parts of the experiment. Our dataset consists of paragraphs taken from open access PubMed papers labeled with rhetorical information as a result of our pilot annotation. Our model is a Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) cells that labels clauses. The clause representations are computed by combining word representations using a…
Narrative is a ubiquitous component of human communication. Understanding its structure plays a critical role in a wide variety of applications, ranging from simple comparative analyses to enhanced narrative retrieval, comprehension, or reasoning capabilities. Prior research in narratology has highlighted the importance of studying the links between cognitive and linguistic aspects of narratives for effective comprehension. This interdependence is related to the textual semantics and mental language in narratives, referring to characters' motivations, feelings or emotions, and beliefs. However…
The University of Melbourne（墨尔本大学）；School of Computing and Information Systems（计算与信息系统学院）
Story comprehension requires a deep semantic understanding of the narrative, making it a challenging task. Inspired by previous studies on ROC Story Cloze Test, we propose a novel method, tracking various semantic aspects with external neural memory chains while encouraging each to focus on a particular semantic aspect. Evaluated on the task of story ending prediction, our model demonstrates superior performance to a collection of competitive baselines, setting a new state of the art.
Princeton University（普林斯顿大学）；New Jersey Institute of Technology（新泽西理工学院）；Syracuse University（雪城大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；Pattern Recognition Center, WeChat AI（微信人工智能模式识别中心）
We propose a new task for assessing machines' skills of understanding fictional characters in narrative stories. The task, TVShowGuess, builds on the scripts of TV series and takes the form of guessing the anonymous main characters based on the backgrounds of the scenes and the dialogues. Our human study supports that this form of task covers comprehension of multiple types of character persona, including understanding characters' personalities, facts and memories of personal experience, which are well aligned with the psychological and literary theories about the theory of mind (ToM) of human…
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Natural language understanding often requires deep semantic knowledge. Expanding on previous proposals, we suggest that some important aspects of semantic knowledge can be modeled as a language model if done at an appropriate level of abstraction. We develop two distinct models that capture semantic frame chains and discourse information while abstracting over the specific mentions of predicates and entities. For each model, we investigate four implementations: a "standard" N-gram language model and three discriminatively trained "neural" language models that generate embeddings for semantic f…
LLMs for automatic annotation of Mandarin narrative transcripts
Shanghai Jiao Tong University（上海交通大学）；University of California San Diego（加州大学圣地亚哥分校）；The Hong Kong Polytechnic University（香港理工大学）；National Research Center for Language and Well-Being（语言与幸福感国家研究中心）；City University of Hong Kong（香港城市大学）
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of Washington（华盛顿大学）
Understanding narratives requires reading between the lines, which in turn, requires interpreting the likely causes and effects of events, even when they are not mentioned explicitly. In this paper, we introduce Cosmos QA, a large-scale dataset of 35,600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions. In stark contrast to most existing reading comprehension datasets where the questions focus on factual and literal understanding of the context paragraph, our dataset focuses on reading between the lines over a diverse collection of people's…
Brown University（布朗大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Sussex（苏塞克斯大学）
We present a novel corpus of 445 human- and computer-generated documents, comprising about 27,000 clauses, annotated for semantic clause types and coherence relations that allow for nuanced comparison of artificial and natural discourse modes. The corpus covers both formal and informal discourse, and contains documents generated using fine-tuned GPT-2 (Zellers et al., 2019) and GPT-3(Brown et al., 2020). We showcase the usefulness of this corpus for detailed discourse analysis of text generation by providing preliminary evidence that less numerous, shorter and more often incoherent clause rela…
Carnegie Mellon University（卡内基梅隆大学）；University of Southern California（南加州大学）
Motivated by recent evidence pointing out the fragility of high-performing span prediction models, we direct our attention to multiple choice reading comprehension. In particular, this work introduces a novel method for improving answer selection on long documents through weighted global normalization of predictions over portions of the documents. We show that applying our method to a span prediction model adapted for answer selection helps model performance on long summaries from NarrativeQA, a challenging reading comprehension dataset with an answer selection task, and we strongly improve on…
Toyota Technological Institute at Chicago（芝加哥丰田工业大学）；University of Chicago（芝加哥大学）
Progress in text understanding has been driven by large datasets that test particular capabilities, like recent datasets for reading comprehension (Hermann et al., 2015). We focus here on the LAMBADA dataset (Paperno et al., 2016), a word prediction task requiring broader context than the immediate sentence. We view LAMBADA as a reading comprehension problem and apply comprehension models based on neural networks. Though these models are constrained to choose a word from the context, they improve the state of the art on LAMBADA from 7.3% to 49%. We analyze 100 instances, finding that neural ne…
From Variance to Invariance: Qualitative Content Analysis for Narrative Graph Annotation
Department of Computer Science, University of Hamburg（汉堡大学计算机科学系）；Department of Socioeconomics, University of Hamburg（汉堡大学社会经济系）；AI and Explanability Group, Leuphana University of Lüneburg（吕内堡大学人工智能与可解释性小组）
Enthymemes are defined as arguments where a premise or conclusion is left implicit. We tackle the task of generating the implicit premise in an enthymeme, which requires not only an understanding of the stated conclusion and premise but also additional inferences that could depend on commonsense knowledge. The largest available dataset for enthymemes (Habernal et al., 2018) consists of 1.7k samples, which is not large enough to train a neural text generation model. To address this issue, we take advantage of a similar task and dataset: Abductive reasoning in narrative text (Bhagavatula et al.,…
Indian Institute of Information Technology, Allahabad（印度理工学院安拉阿巴德分校）
Repeated reading (RR) helps learners, who have little to no experience with reading fluently to gain confidence, speed and process words automatically. The benefits of repeated readings include helping all learners with fact recall, aiding identification of learners' main ideas and vocabulary, increasing comprehension, leading to faster reading as well as increasing word recognition accuracy, and assisting struggling learners as they transition from word-by-word reading to more meaningful phrasing. Thus, RR ultimately helps in improvements of learners' oral fluency and narrative production. Ho…
Question answering (QA) is an important use case on voice assistants. A popular approach to QA is extractive reading comprehension (RC) which finds an answer span in a text passage. However, extractive answers are often unnatural in a conversational context which results in suboptimal user experience. In this work, we investigate conversational answer generation for QA. We propose AnswerBART, an end-to-end generative RC model which combines answer generation from multiple passages with passage ranking and answerability. Moreover, a hurdle in applying generative RC are hallucinations where the …
