Language technologies contribute to promoting multilingualism and linguistic diversity around the world. However, only a very small number of the over 7000 languages of the world are represented in the rapidly evolving language technologies and applications. In this paper we look at the relation between the types of languages, resources, and their representation in NLP conferences to understand the trajectory that different languages have followed over time. Our quantitative investigation underlines the disparity between languages, especially in terms of their resources, and calls into question the "language agnostic" status of current models and systems. Through this paper, we attempt to convince the ACL community to prioritise the resolution of the predicaments highlighted here, so that no language is left behind.
Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
Transfer learning from pre-trained neural language models towards downstream tasks has been a predominant theme in NLP recently. Several researchers have shown that deep NLP models learn non-trivial amount of linguistic knowledge, captured at different layers of the model. We investigate how fine-tuning towards downstream NLP tasks impacts the learned linguistic knowledge. We carry out a study across popular pre-trained models BERT, RoBERTa and XLNet using layer and neuron-level diagnostic classifiers. We found that for some GLUE tasks, the network relies on the core linguistic information and…
University of Maryland（马里兰大学）；University of Washington（华盛顿大学）；Microsoft Research New York（微软纽约研究院）
This paper presents a summary of the first Workshop on Building Linguistically Generalizable Natural Language Processing Systems, and the associated Build It Break It, The Language Edition shared task. The goal of this workshop was to bring together researchers in NLP and linguistics with a shared task aimed at testing the generalizability of NLP systems beyond the distributions of their training data. We describe the motivation, setup, and participation of the shared task, provide discussion of some highlighted results, and discuss lessons learned.
Institute for Logic, Language and Computation（逻辑、语言与计算研究所）；University of Amsterdam（阿姆斯特丹大学）
We investigate the birth and diffusion of lexical innovations in a large dataset of online social communities. We build on sociolinguistic theories and focus on the relation between the spread of a novel term and the social role of the individuals who use it, uncovering characteristics of innovators and adopters. Finally, we perform a prediction task that allows us to anticipate whether an innovation will successfully spread within a community.
NLP's sphere of influence went much beyond computer science research and the development of software applications in the past decade. We see people using NLP methods in a range of academic disciplines from Asian Studies to Clinical Oncology. We also notice the presence of NLP as a module in most of the data science curricula within and outside of regular university setups. These courses are taken by students from very diverse backgrounds. This paper takes a closer look at some issues related to teaching NLP to these diverse audiences based on my classroom experiences, and identifies some chall…
Linguistic disparity in the NLP world is a problem that has been widely acknowledged recently. However, different facets of this problem, or the reasons behind this disparity are seldom discussed within the NLP community. This paper provides a comprehensive analysis of the disparity that exists within the languages of the world. We show that simply categorising languages considering data availability may not be always correct. Using an existing language categorisation based on speaker population and vitality, we analyse the distribution of language data resources, amount of NLP/CL research, in…
Indian Institute of Technology (BHU)（印度理工学院（巴纳拉斯印度大学））
Ideas from forensic linguistics are now being used frequently in Natural Language Processing (NLP), using machine learning techniques. While the role of forensic linguistics was more benign earlier, it is now being used for purposes which are questionable. Certain methods from forensic linguistics are employed, without considering their scientific limitations and ethical concerns. While we take the specific case of forensic linguistics as an example of such trends in NLP and machine learning, the issue is a larger one and present in many other scientific and data-driven domains. We suggest tha…
Recent progress in NLP is driven by pretrained models leveraging massive datasets and has predominantly benefited the world's political and economic superpowers. Technologically underserved languages are left behind because they lack such resources. Hundreds of underserved languages, nevertheless, have available data sources in the form of interlinear glossed text (IGT) from language documentation efforts. IGT remains underutilized in NLP work, perhaps because its annotations are only semi-structured and often language-specific. With this paper, we make the case that IGT data can be leveraged …
Northwestern University（西北大学）；The Hong Kong Polytechnic University（香港理工大学）
Irony is a ubiquitous figurative language in daily communication. Previously, many researchers have approached irony from linguistic, cognitive science, and computational aspects. Recently, some progress have been witnessed in automatic irony processing due to the rapid development in deep neural models in natural language processing (NLP). In this paper, we will provide a comprehensive overview of computational irony, insights from linguistic theory and cognitive science, as well as its interactions with downstream NLP tasks and newly proposed multi-X irony processing perspectives.
Tata Consultancy Services Limited（塔塔咨询服务有限公司）；Yale University（耶鲁大学）；University of Waterloo（滑铁卢大学）
A classification scheme of a scientific subject gives an overview of its body of knowledge. It can also be used to facilitate access to research articles and other materials related to the subject. For example, the ACM Computing Classification System (CCS) is used in the ACM Digital Library search interface and also for indexing computer science papers. We observed that a comprehensive classification system like CCS or Mathematics Subject Classification (MSC) does not exist for Computational Linguistics (CL) and Natural Language Processing (NLP). We propose a classification scheme -- CLICKER f…
Linguistic Data Consortium（语言数据联盟）；University of Pennsylvania（宾夕法尼亚大学）
We investigate the question of whether advances in NLP over the last few years make it possible to vastly increase the size of data usable for research in historical syntax. This brings together many of the usual tools in NLP - word embeddings, tagging, and parsing - in the service of linguistic queries over automatically annotated corpora. We train a part-of-speech (POS) tagger and parser on a corpus of historical English, using ELMo embeddings trained over a billion words of similar text. The evaluation is based on the standard metrics, as well as on the accuracy of the query searches using …
This paper outlines the results of sentence level linguistics based rules for improving part-of-speech tagging. It is well known that the performance of complex NLP systems is negatively affected if one of the preliminary stages is less than perfect. Errors in the initial stages in the pipeline have a snowballing effect on the pipeline's end performance. We have created a set of linguistics based rules at the sentence level which adjust part-of-speech tags from state-of-the-art taggers. Comparison with state-of-the-art taggers on widely used benchmarks demonstrate significant improvements in t…
There is an ongoing debate in the NLP community whether modern language models contain linguistic knowledge, recovered through so-called probes. In this paper, we study whether linguistic knowledge is a necessary condition for the good performance of modern language models, which we call the \textit{rediscovery hypothesis}. In the first place, we show that language models that are significantly compressed but perform well on their pretraining objectives retain good scores when probed for linguistic structures. This result supports the rediscovery hypothesis and leads to the second contribution…
Linguistic ambiguity is and has always been one of the main challenges in Natural Language Processing (NLP) systems. Modern Transformer architectures like BERT, T5 or more recently InstructGPT have achieved some impressive improvements in many NLP fields, but there is still plenty of work to do. Motivated by the uproar caused by ChatGPT, in this paper we provide an introduction to linguistic ambiguity, its varieties and their relevance in modern NLP, and perform an extensive empiric analysis. ChatGPT strengths and weaknesses are revealed, as well as strategies to get the most of this model.
BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resources
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India（印度理工学院帕纳加邦计算机科学与工程系）；Indian Institute of Technology Patna, India（印度理工学院帕纳加邦）
Teaching NLP with Bracelets and Restaurant Menus: An Interactive Workshop for Italian Students
University of Trento（特伦托大学）；Aston University（阿斯顿大学）；University of Bologna（博洛尼亚大学）；University of Pisa（比萨大学）；University of Trieste（的里雅斯特大学）；University of Groningen（格罗宁根大学）
In this article, we propose a Category Theory approach to (syntactic) interoperability between linguistic tools. The resulting category consists of textual documents, including any linguistic annotations, NLP tools that analyze texts and add additional linguistic information, and format converters. Format converters are necessary to make the tools both able to read and to produce different output formats, which is the key to interoperability. The idea behind this document is the parallelism between the concepts of composition and associativity in Category Theory with the NLP pipelines. We show…
Recent trends in NLP research have raised an interest in linguistic code-switching (CS); modern approaches have been proposed to solve a wide range of NLP tasks on multiple language pairs. Unfortunately, these proposed methods are hardly generalizable to different code-switched languages. In addition, it is unclear whether a model architecture is applicable for a different task while still being compatible with the code-switching setting. This is mainly because of the lack of a centralized benchmark and the sparse corpora that researchers employ based on their specific needs and interests. To …
LTL, University of Cambridge（剑桥大学LTL（语言与理论语言学实验室））；Department of Brain and Cognitive Sciences, MIT（麻省理工学院脑与认知科学系）；Faculty of Industrial Engineering and Management, Technion – IIT（以色列理工学院工业工程与管理学院）；LATTICE Lab, CNRS and ENS/PSL and Univ. Sorbonne nouvelle/USPC（LATTICE实验室（法国国家科学研究中心、巴黎高等师范学院/PSL、巴黎索邦大学/巴黎文理研究大学））；ILLC, University of Amsterdam（阿姆斯特丹大学ILLC（语言、逻辑与计算研究所））
Linguistic typology aims to capture structural and semantic variation across the world's languages. A large-scale typology could provide excellent guidance for multilingual Natural Language Processing (NLP), particularly for languages that suffer from the lack of human labeled resources. We present an extensive literature survey on the use of typological information in the development of NLP techniques. Our survey demonstrates that to date, the use of information in existing typological databases has resulted in consistent but modest improvements in system performance. We show that this is due…
Qatar Computing Research Institute, HBKU Research Complex（卡塔尔计算研究所，哈马德·本·哈利法大学研究中心）；MIT Computer Science and Artificial Intelligence Laboratory（麻省理工学院计算机科学与人工智能实验室）
Despite the remarkable evolution of deep neural networks in natural language processing (NLP), their interpretability remains a challenge. Previous work largely focused on what these models learn at the representation level. We break this analysis down further and study individual dimensions (neurons) in the vector representation learned by end-to-end neural models in NLP tasks. We propose two methods: Linguistic Correlation Analysis, based on a supervised method to extract the most relevant neurons with respect to an extrinsic task, and Cross-model Correlation Analysis, an unsupervised method…
A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions
MBZUAI（Mohamed bin Zayed 人工智能大学）；German International University, Egypt（德国国际大学）；University of Stuttgart（斯图加特大学）；University of Houston（休斯顿大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）
Actionable sentences are terms that, in the most basic sense, imply the necessity of taking a specific action. In Linguistic terms, they are steps to achieve an operation, often through the usage of action verbs. For example, the sentence, `Get your homework finished by tomorrow` qualifies as actionable since it demands a specific action (In this case, finishing homework) to be taken. In contrast, a simple sentence such as, `I like to play the guitar` does not qualify as an actionable phrase since it simply states a personal choice of the person instead of demanding a task to be finished. In t…
Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead
Saarland University, Saarland Informatics Campus（萨尔兰大学，萨尔兰信息学校区）；LMU Munich and Munich Center for Machine Learning（慕尼黑大学及慕尼黑机器学习中心）；Mila - Quebec AI Institute, McGill University & Canada CIFAR AI Chair（魁北克人工智能研究所，麦吉尔大学及加拿大 CIFAR 人工智能 chair）
Carnegie Mellon University（卡内基梅隆大学）；National Research Council Canada（加拿大国家研究委员会）
Language documentation is inherently a time-intensive process; transcription, glossing, and corpus management consume a significant portion of documentary linguists' work. Advances in natural language processing can help to accelerate this work, using the linguists' past decisions as training material, but questions remain about how to prioritize human involvement. In this extended abstract, we describe the beginnings of a new project that will attempt to ease this language documentation process through the use of natural language processing (NLP) technology. It is based on (1) methods to adap…
The natural language system DISCO is described. It combines o a powerful and flexible grammar development system; o linguistic competence for German including morphology, syntax and semantics; o new methods for linguistic performance modelling on the basis of high-level competence grammars; o new methods for modelling multi-agent dialogue competence; o an interesting sample application for appointment scheduling and calendar management.
North Africa's Missing Framework: NLP-Driven Mental Healthcare in Algeria and Implications for Low-resource Settings
University Mohamed El Bachir El Ibrahimi of Bordj Bou Arreridj（布吉布阿雷里吉·穆罕默德·巴希尔·易卜拉欣大学）；Laboratory of Informatics and its Applications of M’sila（姆西拉信息学及其应用实验室）；Laboratory of Materials and Electronic Systems（材料与电子系统实验室）
Natural Language Processing offers new insights into language data across almost all disciplines and domains, and allows us to corroborate and/or challenge existing knowledge. The primary hurdles to widening participation in and use of these new research tools are, first, a lack of coding skills in students across K-16, and in the population at large, and second, a lack of knowledge of how NLP-methods can be used to answer questions of disciplinary interest outside of linguistics and/or computer science. To broaden participation in NLP and improve NLP-literacy, we introduced a new tool web-bas…
Multilingual sentence encoders are widely used to transfer NLP models across languages. The success of this transfer is, however, dependent on the model's ability to encode the patterns of cross-lingual similarity and variation. Yet, little is known as to how these models are able to do this. We propose a simple method to study how relationships between languages are encoded in two state-of-the-art multilingual models (i.e. M-BERT and XLM-R). The results provide insight into their information sharing mechanisms and suggest that linguistic properties are encoded jointly across typologically-sim…
University of Copenhagen（哥本哈根大学）；University of Trento（特伦托大学）；Uppsala University（乌普萨拉大学）；KU Leuven（鲁汶大学）；University of Sheffield（谢菲尔德大学）
Various efforts in the Natural Language Processing (NLP) community have been made to accommodate linguistic diversity and serve speakers of many different languages. However, it is important to acknowledge that speakers and the content they produce and require, vary not just by language, but also by culture. Although language and culture are tightly linked, there are important differences. Analogous to cross-lingual and multilingual NLP, cross-cultural and multicultural NLP considers these differences in order to better serve users of NLP systems. We propose a principled framework to frame the…
Ubiquitous Knowledge Processing Lab (UKP-TUDA)（普适知识处理实验室（UKP-TUDA））；Hessian Center for AI (hessian.AI)（黑森人工智能中心（hessian.AI））；Technical University of Darmstadt（达姆施塔特工业大学）
University of California, Santa Barbara（加州大学圣巴巴拉分校）
Convolutional neural networks have been successfully applied to various NLP tasks. However, it is not obvious whether they model different linguistic patterns such as negation, intensification, and clause compositionality to help the decision-making process. In this paper, we apply visualization techniques to observe how the model can capture different linguistic features and how these features can affect the performance of the model. Later on, we try to identify the model errors and their sources. We believe that interpreting CNNs is the first step to understand the underlying semantic featur…
Amazon（亚马逊）；Bloomberg（彭博）；The University of Melbourne（墨尔本大学）；HKUST（香港科技大学）；Telkom University（电信大学）；INACL（印尼计算语言学协会）；Universitas Indonesia（印度尼西亚大学）；Kanda University of International Studies（神田外国语大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Google Research（谷歌研究院）
NLP research is impeded by a lack of resources and awareness of the challenges presented by underrepresented languages and dialects. Focusing on the languages spoken in Indonesia, the second most linguistically diverse and the fourth most populous nation of the world, we provide an overview of the current state of NLP research for Indonesia's 700+ languages. We highlight challenges in Indonesian NLP and how these affect the performance of current NLP systems. Finally, we provide general recommendations to help develop NLP technology not only for languages of Indonesia but also other underrepre…
A key aim of science is explanation, yet the idea of explaining language phenomena has taken a backseat in mainstream Natural Language Processing (NLP) and many other areas of Artificial Intelligence. I argue that explanation of linguistic behaviour should be a main goal of NLP, and that this is not the same as making NLP models explainable. To illustrate these ideas, some recent models of human language production are compared with each other. I conclude by asking what it would mean for NLP research and institutional policies if our community took explanatory value seriously, while heeding so…
Georgetown University（乔治城大学）；San Mateo High School（圣马特奥高中）；University of North Carolina – Chapel Hill（北卡罗来纳大学教堂山分校）；Microsoft（微软）
South Asia is home to a plethora of languages, many of which severely lack access to new language technologies. This linguistic diversity also results in a research environment conducive to the study of comparative, contact, and historical linguistics -- fields which necessitate the gathering of extensive data from many languages. We claim that data scatteredness (rather than scarcity) is the primary obstacle in the development of South Asian language technology, and suggest that the study of language history is uniquely aligned with surmounting this obstacle. We review recent developments in …
