Academic research is an exploration activity to solve problems that have never been resolved before. By this nature, each academic research work is required to perform a literature review to distinguish its novelties that have not been addressed by prior works. In natural language processing, this literature review is usually conducted under the "Related Work" section. The task of automatic related work generation aims to automatically generate the "Related Work" section given the rest of the research paper and a list of cited papers. Although this task was proposed over 10 years ago, it received little attention until very recently, when it was cast as a variant of the scientific multi-document summarization problem. However, even today, the problems of automatic related work and citation text generation are not yet standardized. In this survey, we conduct a meta-study to compare the existing literature on related work generation from the perspectives of problem formulation, dataset collection, methodological approach, performance evaluation, and future prospects to provide the reader insight into the progress of the state-of-the-art studies, as well as and how future studies can be conducted. We also survey relevant fields of study that we suggest future work to consider integrating.
In this paper, we propose a method to automatically identify future events in Lebanon's economy from Arabic texts. Challenges are threefold: first, we need to build a corpus of Arabic texts that covers Lebanon's economy; second, we need to study how future events are expressed linguistically in these texts; and third, we need to automatically identify the relevant textual segments accordingly. We will validate this method on a constructed corpus form the web and show that it has very promising results. To do so, we will be using SLCSAS, a system for semantic analysis, based on the Contextual E…
Creating multiple-choice questions to assess reading comprehension of a given article involves generating question-answer pairs (QAPs) and adequate distractors. We present two methods to tackle the challenge of QAP generations: (1) A deep-learning-based end-to-end question generation system based on T5 Transformer with Preprocessing and Postprocessing Pipelines (TP3). We use the finetuned T5 model for our downstream task of question generation and improve accuracy using a combination of various NLP tools and algorithms in preprocessing and postprocessing to select appropriate answers and filte…
Dr. Bhim Rao Ambedkar University（比姆·拉奥·安贝德卡尔博士大学）；Jadavpur University（贾达普大学）；Rutgers University（罗格斯大学）；Jawaharlal Nehru University（贾瓦哈拉尔·尼赫鲁大学）
In this paper, we discuss an attempt to develop an automatic language identification system for 5 closely-related Indo-Aryan languages of India, Awadhi, Bhojpuri, Braj, Hindi and Magahi. We have compiled a comparable corpora of varying length for these languages from various resources. We discuss the method of creation of these corpora in detail. Using these corpora, a language identification system was developed, which currently gives state of the art accuracy of 96.48\%. We also used these corpora to study the similarity between the 5 languages at the lexical level, which is the first data-b…
University of Pennsylvania（宾夕法尼亚大学）；Emory University（埃默里大学）
With the increasing use of social media data for health-related research, the credibility of the information from this source has been questioned as the posts may originate from automated accounts or "bots". While automatic bot detection approaches have been proposed, there are none that have been evaluated on users posting health-related information. In this paper, we extend an existing bot detection system and customize it for health-related research. Using a dataset of Twitter users, we first show that the system, which was designed for political bot detection, underperforms when applied to…
National Institute of Information and Communications Technology（日本国立信息通信技术研究所）
This paper presents the first large-scale meta-evaluation of machine translation (MT). We annotated MT evaluations conducted in 769 research papers published from 2010 to 2020. Our study shows that practices for automatic MT evaluation have dramatically changed during the past decade and follow concerning trends. An increasing number of MT evaluations exclusively rely on differences between BLEU scores to draw conclusions, without performing any kind of statistical significance testing nor human evaluation, while at least 108 metrics claiming to be better than BLEU have been proposed. MT evalu…
Automatic short answer grading is an important research direction in the exploration of how to use artificial intelligence (AI)-based tools to improve education. Current state-of-the-art approaches use neural language models to create vectorized representations of students responses, followed by classifiers to predict the score. However, these approaches have several key limitations, including i) they use pre-trained language models that are not well-adapted to educational subject domains and/or student-generated text and ii) they almost always train one model per question, ignoring the linkag…
Communication is now a standard tool in the central bank's monetary policy toolkit. Theoretically, communication provides the central bank an opportunity to guide public expectations, and it has been shown empirically that central bank communication can lead to financial market fluctuations. However, there has been little research into which dimensions or topics of information are most important in causing these fluctuations. We develop a semi-automatic methodology that summarizes the FOMC statements into its main themes, automatically selects the best model based on coherency, and assesses wh…
Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond English
HiTZ Center, University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU）HiTZ中心）；University of Alicante (UA)（阿利坎特大学（UA））；GOI Institute, Basque Summer University (UEU)（巴斯克夏季大学（UEU）GOI研究所）
Huge amounts of digital videos are being produced and broadcast every day, leading to giant media archives. Effective techniques are needed to make such data accessible further. Automatic meta-data labelling of broadcast media is an essential task for multimedia indexing, where it is standard to use multi-modal input for such purposes. This paper describes a novel method for automatic detection of media genre and show identities using acoustic features, textual features or a combination thereof. Furthermore the inclusion of available meta-data, such as time of broadcast, is shown to lead to ve…
Timely analysis of cyber-security information necessitates automated information extraction from unstructured text. While state-of-the-art extraction methods produce extremely accurate results, they require ample training data, which is generally unavailable for specialized applications, such as detecting security related entities; moreover, manual annotation of corpora is very costly and often not a viable solution. In response, we develop a very precise method to automatically label text from several data sources by leveraging related, domain-specific, structured data and provide public acce…
With the rise in popularity of public social media and micro-blogging services, most notably Twitter, the people have found a venue to hear and be heard by their peers without an intermediary. As a consequence, and aided by the public nature of Twitter, political scientists now potentially have the means to analyse and understand the narratives that organically form, spread and decline among the public in a political campaign. However, the volume and diversity of the conversation on Twitter, combined with its noisy and idiosyncratic nature, make this a hard task. Thus, advanced data mining and…
Indian Statistical Institute（印度统计研究所）；Calcutta University（加尔各答大学）
There is enormous growth in various fields of research. This development is accompanied by new problems. To solve these problems efficiently and in an optimized manner, algorithms are created and described by researchers in the scientific literature. Scientific algorithms are vital for understanding and reusing existing work in numerous domains. However, algorithms are generally challenging to find. Also, the comparison among similar algorithms is difficult because of the disconnected documentation. Information about algorithms is mostly present in websites, code comments, and so on. There is …
Meow: End-to-End Outline Writing for Automatic Academic Survey
Beijing Wenge Technology Co.,Ltd（北京文格科技有限公司）；Peking University（北京大学）；Duke University（杜克大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
