The Chinese University of Hong Kong（香港中文大学）；Microsoft Research, Redmond（微软研究院（雷德蒙德））；Center for Theoretical Physics, Massachusetts Institute of Technology（麻省理工学院理论物理中心）
Deng Cai, Yizhe Zhang, Yichen Huang, Wai Lam, Bill Dolan
We propose the task of narrative incoherence detection as a new arena for inter-sentential semantic understanding: Given a multi-sentence narrative, decide whether there exist any semantic discrepancies in the narrative flow. Specifically, we focus on the missing sentence and discordant sentence detection. Despite its simple setup, this task is challenging as the model needs to understand and analyze a multi-sentence narrative, and predict incoherence at the sentence level. As an initial step towards this task, we implement several baselines either directly analyzing the raw text (\textit{token-level}) or analyzing learned sentence representations (\textit{sentence-level}). We observe that while token-level modeling has better performance when the input contains fewer sentences, sentence-level modeling performs better on longer narratives and possesses an advantage in efficiency and flexibility. Pre-training on large-scale data and auxiliary sentence prediction training objective further boost the detection performance of the sentence-level model.
Mary, the Cheeseburger-Eating Vegetarian: Do LLMs Recognize Incoherence in Narratives?
Department of Computer Science and Engineering, University of Minnesota（计算机科学与工程系，明尼苏达大学）；Department of Educational Psychology, University of Minnesota（教育心理学系，明尼苏达大学）；Department of Linguistics, Hamline University（语言学系，哈姆林大学）；Department of English, University of Minnesota（英语系，明尼苏达大学）
Universität des Saarlandes（萨尔大学）；Universität Stuttgart（斯图加特大学）
Script knowledge consists of detailed information on everyday activities. Such information is often taken for granted in text and needs to be inferred by readers. Therefore, script knowledge is a central component to language comprehension. Previous work on representing scripts is mostly based on extensive manual work or limited to scenarios that can be found with sufficient redundancy in large corpora. We introduce the task of scenario detection, in which we identify references to scripts. In this task, we address a wide range of different scripts (200 scenarios) and we attempt to identify al…
Progress in summarizing long texts is inhibited by the lack of appropriate evaluation frameworks. When a long summary must be produced to appropriately cover the facets of that text, that summary needs to present a coherent narrative to be understandable by a reader, but current automatic and human evaluation methods fail to identify gaps in coherence. In this work, we introduce SNaC, a narrative coherence evaluation framework rooted in fine-grained annotations for long summaries. We develop a taxonomy of coherence errors in generated narrative summaries and collect span-level annotations for …
Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models
Department of Statistics, TU Dortmund University（统计系，多特蒙德大学）；Institute of Journalism, TU Dortmund University（新闻学院，多特蒙德大学）；Faculty of Management and Economics, Ruhr University Bochum（管理与经济学院，波恩大学）
NLP-based detection of systematic anomalies among the narratives of consumer complaints
School of Mathematical and Statistical Sciences, Western University（西安大略大学数学与统计学院）；Agri-Food Analytics Lab, Dalhousie University（达尔豪斯大学农业食品分析实验室）；Canada Life（加拿大永明金融）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；Institute for Health Care Delivery Science, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院医疗服务科学研究所）
ExTax: Explainable Disinformation Detection via Persuasion, Emotion, and Narrative Role Taxonomies
Peking University（北京大学）；University of Science and Technology of China（中国科学技术大学）；North China University of Science and Technology（华北理工大学）；Tsinghua University（清华大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Chinese Academy of Sciences（中国科学院大学）；Soochow University（苏州大学）；Beihang University（北航）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Karolinska Institutet（卡罗林斯卡学院）；Sahlgrenska University Hospital（萨尔格伦斯卡大学医院）
Accurate and timely detection of medical adverse events (AEs) from free-text medical narratives is challenging. Natural language processing (NLP) with deep learning has already shown great potential for analyzing free-text data, but its application for medical AE detection has been limited. In this study we proposed deep learning based NLP (DL-NLP) models for efficient and accurate hip dislocation AE detection following total hip replacement from standard (radiology notes) and non-standard (follow-up telephone notes) free-text medical narratives. We benchmarked these proposed models with a wid…
Retrieval-Augmented Multimodal Model for Fake News Detection
University of International Business and Economics（国际商务经济大学）；Peking University（北京大学）；University of Southern California（南加州大学）；Upstart Holdings, Inc.（Upstart Holdings公司）
Conflicts, Villains, Resolutions: Towards models of Narrative Media Framing
The University of Melbourne（墨尔本大学）；The Hong Kong Polytechnic University（香港理工大学）；CSIRO Data61（澳大利亚联邦科学与工业研究组织Data61）；The Australian National University（澳大利亚国立大学）
Authorship analysis is an important subject in the field of natural language processing. It allows the detection of the most likely writer of articles, news, books, or messages. This technique has multiple uses in tasks related to authorship attribution, detection of plagiarism, style analysis, sources of misinformation, etc. The focus of this paper is to explore the limitations and sensitiveness of established approaches to adversarial manipulations of inputs. To this end, and using those established techniques, we first developed an experimental frame-work for author detection and input pert…
Rumors and conspiracy theories thrive in environments of low confidence and low trust. Consequently, it is not surprising that ones related to the Covid-19 pandemic are proliferating given the lack of any authoritative scientific consensus on the virus, its spread and containment, or on the long term social and economic ramifications of the pandemic. Among the stories currently circulating are ones suggesting that the 5G network activates the virus, that the pandemic is a hoax perpetrated by a global cabal, that the virus is a bio-weapon released deliberately by the Chinese, or that Bill Gates…
This paper is motivated by the automation of neuropsychological tests involving discourse analysis in the retellings of narratives by patients with potential cognitive impairment. In this scenario the task of sentence boundary detection in speech transcripts is important as discourse analysis involves the application of Natural Language Processing tools, such as taggers and parsers, which depend on the sentence as a processing unit. Our aim in this paper is to verify which embedding induction method works best for the sentence boundary detection task, specifically whether it be those which wer…
Detecting Narrative Shifts through Persistent Structures: A Topological Analysis of Media Discourse
Department of Cyber Intelligence and Data Science（网络智能与数据科学系）；Biological and Computational Intelligence Center（生物与计算智能中心）；National Intelligence University（国家智能大学）
Technische Hochschule Nürnberg Georg Simon Ohm（纽伦堡乔治·西蒙·欧姆应用科学大学）；University of Trento（特伦托大学）
Personal narratives (PN) - spoken or written - are recollections of facts, people, events, and thoughts from one's own experience. Emotion recognition and sentiment analysis tasks are usually defined at the utterance or document level. However, in this work, we focus on Emotion Carriers (EC) defined as the segments (speech or text) that best explain the emotional state of the narrator ("loss of father", "made me choose"). Once extracted, such EC can provide a richer representation of the user state to improve natural language understanding and dialogue modeling. In previous work, it has been s…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Compromised accounts on social networks are regular user accounts that have been taken over by an entity with malicious intent. Since the adversary exploits the already established trust of a compromised account, it is crucial to detect these accounts to limit the damage they can cause. We propose a novel general framework for semantic analysis of text messages coming out from an account to detect compromised accounts. Our framework is built on the observation that normal users will use language that is measurably different from the language that an adversary would use when the account is comp…
In the era of clinical information explosion, a good strategy for clinical text summarization is helpful to improve the clinical workflow. The ideal summarization strategy can preserve important information in the informative but less organized, ill-structured clinical narrative texts. Instead of using pure statistical learning approaches, which are difficult to interpret and explain, we utilized knowledge of computational linguistics with human experts-curated biomedical knowledge base to achieve the interpretable and meaningful clinical text summarization. Our research objective is to use th…
