自然语言处理的统计物理学

迈向高级语音信号处理：基于卷积的架构及其应用的统计视角

Towards Advanced Speech Signal Processing: A Statistical Perspective on Convolution-Based Architectures and its Applications

本文综述基于卷积的语音信号处理模型（如CNNs、Conformers等）的统计背景与多场景应用，通过对比训练成本、模型大小等指标分析各模型优劣，为语音技术发展提供研究方向。

ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing

Despite recent advances in natural language processing, many statistical models for processing text perform extremely poorly under domain shift. Processing biomedical and clinical text is a critically important application area of natural language processing, for which there are few robust, practical, publicly available models. This paper describes scispaCy, a new tool for practical biomedical/scientific text processing, which heavily leverages the spaCy library. We detail the performance of two packages of models released in scispaCy and demonstrate their robustness on several tasks and datas…

自然语言处理的神经网络模型入门

A Primer on Neural Network Models for Natural Language Processing

在过去的几年里，神经网络已经重新成为强大的机器学习模型，在图像识别和语音处理等领域取得了最先进的成果。最近，神经网络模型也开始应用于文本自然语言信号，再次取得了非常有希望的结果。本教程从自然语言处理研究的角度调查神经网络模型，试图使自然语言研究人员加快神经技术的速度。本教程涵盖了自然语言任务的输入编码、前馈网络、卷积网络、递归网络和递归网络，以及自动梯度计算的计算图抽象。

基于自然语言处理的民间故事信息检索

Information retrieval in folktales using natural language processing

我们的目标是从非结构化文本中提取文学人物的信息。我们对领域本体进行自然语言处理和推理。第一个任务是确定主要人物和故事中这些人物被描述或表演的部分。我们说明了在民间故事领域的一个场景中的系统。该系统依赖于一个民间故事本体，我们已经开发的基础上Propp的模型的民间故事形态。

Deep Learning Based Natural Language Processing for End to End Speech Translation

Deep Learning methods employ multiple processing layers to learn hierarchial representations of data. They have already been deployed in a humongous number of applications and have produced state-of-the-art results. Recently with the growth in processing power of computers to be able to do high dimensional tensor calculations, Natural Language Processing (NLP) applications have been given a significant boost in terms of efficiency as well as accuracy. In this paper, we will take a look at various signal processing techniques and then application of them to produce a speech-to-text system using…

Thoth: Improved Rapid Serial Visual Presentation using Natural Language Processing

Thoth is a tool designed to combine many different types of speed reading technology. The largest insight is using natural language parsing for more optimal rapid serial visual presentation and more effective reading information.

使用DIFF命令进行自然语言处理

Using the DIFF Command for Natural Language Processing

Diff是一个软件程序，用于检测两个数据集之间的差异，在自然语言处理中很有用。本文给出了几个应用差异的例子。它们包括两个不同数据集之间的差异检测，重写规则的提取，两个不同数据集的合并，以及两个不同数据集的最佳匹配。由于diff随任何标准UNIX系统一起提供，因此它随时可用且非常易于使用。我们的研究表明diff是研究自然语言处理的一个实用工具。

通过复杂网络的拓扑分析展望自然语言处理任务的发展

A perspective on the advancement of natural language processing tasks via topological analysis of complex networks

对Cong和Liu的"Approaching human language with complex networks"的评论（Physics of Life Reviews，Volume 11，Issue 4，December 2014，Pages 598 - 618）。

Representation Learning for Natural Language Processing

This book aims to review and present the recent advances of distributed representation learning for NLP, including why representation learning can improve NLP, how representation learning takes part in various important topics of NLP, and what challenges are still not well addressed by distributed representation.

Problems and Countermeasures in Natural Language Processing Evaluation

Evaluation in natural language processing guides and promotes research on models and methods. In recent years, new evalua-tion data sets and evaluation tasks have been continuously proposed. At the same time, a series of problems exposed by ex-isting evaluation have also restricted the progress of natural language processing technology. Starting from the concept, com-position, development and meaning of natural language evaluation, this article classifies and summarizes the tasks and char-acteristics of mainstream natural language evaluation, and then summarizes the problems and causes of natu…

Categorical Tools for Natural Language Processing

This thesis develops the translation between category theory and computational linguistics as a foundation for natural language processing. The three chapters deal with syntax, semantics and pragmatics. First, string diagrams provide a unified model of syntactic structures in formal grammars. Second, functors compute semantics by turning diagrams into logical, tensor, neural or quantum computation. Third, the resulting functorial models can be composed to form games where equilibria are the solutions of language processing tasks. This framework is implemented as part of DisCoPy, the Python lib…

将AI规划与自然语言处理相结合：显性知识与隐性知识的融合

Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledge

本文首次系统综述AI规划与自然语言处理的深层联系，提出二者通过基于规划的文本理解、NLP、可解释性、人机交互及应用五个方面相互促进，融合显性与隐性知识以改善人机通信。

使用自然语言处理的自主需求规格说明处理

Autonomous requirements specification processing using natural language processing

我们描述了我们正在进行的研究，集中在自然语言处理（NLP）的软件工程和系统开发活动的应用。特别是，本文讨论了在需求分析和系统设计过程中使用NLP。我们已经开发了一个原型工具集，可以帮助系统分析师或软件工程师选择和验证与项目相关的术语。在本文中，我们描述的过程中所采用的系统提取和分类的兴趣对象的需求文件。这些过程是用一个小例子来说明。

A Survey of Active Learning for Natural Language Processing

In this work, we provide a survey of active learning (AL) for its applications in natural language processing (NLP). In addition to a fine-grained categorization of query strategies, we also investigate several other important aspects of applying AL to NLP problems. These include AL for structured prediction tasks, annotation cost, model learning (especially with deep neural models), and starting and stopping AL. Finally, we conclude with a discussion of related topics and future directions.

“I'm sorry Dave，I'm afraid I can't do that”：Linguistics，Statistics，and自然语言处理circa 2001

"I'm sorry Dave, I'm afraid I can't do that": Linguistics, Statistics, and Natural Language Processing circa 2001

对自然语言处理历史的简要概述，重点是数据驱动的方法。主题包括“歧义和语言分析”，“Firth things first”，“A 'C' change”和“The Questicists Strike Back”。

Resources for Turkish Natural Language Processing: A critical survey

This paper presents a comprehensive survey of corpora and lexical resources available for Turkish. We review a broad range of resources, focusing on the ones that are publicly available. In addition to providing information about the available linguistic resources, we present a set of recommendations, and identify gaps in the data available for conducting research and building applications in Turkish Linguistics and Natural Language Processing.

Reasoning about Procedures with Natural Language Processing: A Tutorial

This tutorial provides a comprehensive and in-depth view of the research on procedures, primarily in Natural Language Processing. A procedure is a sequence of steps intended to achieve some goal. Understanding procedures in natural language has a long history, with recent breakthroughs made possible by advances in technology. First, we discuss established approaches to collect procedures, by human annotation or extraction from web resources. Then, we examine different angles from which procedures can be reasoned about, as well as ways to represent them. Finally, we enumerate scenarios where pr…

Ad Text Classification with Transformer-Based Natural Language Processing Methods

In this study, a natural language processing-based (NLP-based) method is proposed for the sector-wise automatic classification of ad texts created on online advertising platforms. Our data set consists of approximately 21,000 labeled advertising texts from 12 different sectors. In the study, the Bidirectional Encoder Representations from Transformers (BERT) model, which is a transformer-based language model that is recently used in fields such as text classification in the natural language processing literature, was used. The classification efficiencies obtained using a pre-trained BERT model …

Hitachi at SemEval-2020 Task 12: Offensive Language Identification with Noisy Labels using Statistical Sampling and Post-Processing

In this paper, we present our participation in SemEval-2020 Task-12 Subtask-A (English Language) which focuses on offensive language identification from noisy labels. To this end, we developed a hybrid system with the BERT classifier trained with tweets selected using Statistical Sampling Algorithm (SA) and Post-Processed (PP) using an offensive wordlist. Our developed system achieved 34 th position with Macro-averaged F1-score (Macro-F1) of 0.90913 over both offensive and non-offensive classes. We further show comprehensive results and error analysis to assist future research in offensive lan…

New Approaches for Natural Language Understanding based on the Idea that Natural Language encodes both Information and its Processing Procedures

We must recognize that natural language is a way of information encoding, and it encodes not only the information but also the procedures for how information is processed. To understand natural language, the same as we conceive and design computer languages, the first step is to separate information (or data) and the processing procedures of information (or data). In natural language, some processing procedures of data are encoded directly as the structure chunk and the pointer chunk (this paper has reclassified lexical chunks as the data chunk, structure chunk, and the pointer chunk); some pr…

自然语言处理：技术现状、当前趋势和挑战

Natural Language Processing: State of The Art, Current Trends and Challenges

自然语言处理（NLP）近年来因其对人类语言的计算机表示和分析而受到广泛关注。它已经在机器翻译、垃圾邮件检测、信息抽取、摘要、医学、问答等领域得到了广泛的应用。本文通过讨论自然语言处理的不同层次和自然语言生成（NLG）的组成部分，将NLP分为四个阶段，介绍了NLP的各种应用以及当前的趋势和挑战。

Suggesting Relevant Questions for a Query Using Statistical Natural Language Processing Technique

Suggesting similar questions for a user query has many applications ranging from reducing search time of users on e-commerce websites, training of employees in companies to holistic learning for students. The use of Natural Language Processing techniques for suggesting similar questions is prevalent over the existing architecture. Mainly two approaches are studied for finding text similarity namely syntactic and semantic, however each has its draw-backs and fail to provide the desired outcome. In this article, a self-learning combined approach is proposed for determining textual similarity tha…

自然语言处理：结构与复杂性

Natural Language Processing: Structure and Complexity

本文介绍了一种分析自然语言处理任务复杂度的方法，以及预测新的自然语言处理任务难度的方法。我们的复杂性措施来自Kolmogorov复杂性的一类自动机- { \it的意思是自动机}，其目的是提取相关的信息片段的句子。自然语言语义的定义仅与自动机可以回答的问题集相关。本文展示了各种NLP程序和任务的复杂性估计的例子，以及复杂性管理的一些方法。它将自然语言处理定位为软件工程的一个子领域，并奠定了其形式化的基础。

Panini梵文语法的模糊建模与自然语言处理

Fuzzy Modeling and Natural Language Processing for Panini's Sanskrit Grammar

印度语在世界自然语言中有着悠久的历史。帕尼尼是第一个在五世纪用大约4000条规则定义梵语语法的人。这些规则包含不确定性信息。用计算机处理具有不确定信息的梵语是不可能的。本文提出用模糊逻辑和模糊推理来处理梵语文法推理中的不确定性信息。本文还讨论了梵文的处理。

基于规则的文件映射DBMS和自然语言处理

File mapping Rule-based DBMS and Natural Language Processing

本文描述了一种类似于自然语言结构的信息存储、提取和处理系统。对于递归推理，系统使用具有相同表示的规则作为数据。操作系统的文件映射（SHM）机制为信息的存储提供了环境。本文阐述了动态数据结构和推理规则记录语言的构造原则，考虑了现有实现的特点，并给出了在自然语言上实现语义信息检索的应用描述。

用于统计语言处理的聚合和混合阶马尔可夫模型

Aggregate and mixed-order Markov models for statistical language processing

我们考虑使用的语言模型，其大小和准确性是中间不同的顺序n-gram模型。特别研究了两种类型的模型。聚合马尔可夫模型是基于类的二元模型，其中从词到类的映射是概率性的。混合阶马尔可夫模型结合了二元模型，其预测以不同的单词为条件。这两种类型的模型训练的期望最大化（EM）算法的极大似然估计。我们研究了将这些模型插入不同阶n元语法之间的平滑过程。这被发现显着减少看不见的单词组合的困惑。

A Tidy Data Model for Natural Language Processing using cleanNLP

The package cleanNLP provides a set of fast tools for converting a textual corpus into a set of normalized tables. The underlying natural language processing pipeline utilizes Stanford's CoreNLP library, exposing a number of annotation tasks for text written in English, French, German, and Spanish. Annotators include tokenization, part of speech tagging, named entity recognition, entity linking, sentiment analysis, dependency parsing, coreference resolution, and information extraction.

本体论在自然语言处理中的理论地位

The Theoretical Status of Ontologies in Natural Language Processing

本文讨论了“本体论”在自然语言处理中的应用。它分类了各种类型的本体，已在自然语言处理，并讨论了各种好处和问题，这些设计。然后，特别关注的是在使用上模型，一个语言的动机“本体”最初设计用于与Penman文本生成系统中获得的经验。进一步的NLP本体设计标准的一些建议。

金融监管中的自然语言处理

Natural Language Processing for Financial Regulation

本文探讨金融监管中无监督数据集下的语义匹配，提出利用免费资源超越预训练句子转换器，并解释NLP核心数学原理。

用于自然语言处理的半监督分类

Semi-supervised Classification for Natural Language Processing

半监督分类是一个有趣的想法，其中分类模型是从标记和未标记的数据中学习的。在自然语言处理领域，它与监督分类相比有许多优点。例如，监督分类只利用昂贵的标记数据，这些数据通常难以获得，数量不足，并且需要人类专家进行注释。另一方面，未标记的数据便宜且丰富。尽管许多因素限制了半监督分类的广泛使用，但由于其性能水平在经验上与监督分类一样好，因此它已经变得流行。本研究探讨了半监督分类在句法分析、生物医学信息处理、文本分类和摘要等自然语言处理任务中的可能性和成就以及复杂性和局限性。

Natural Language Processing in-and-for Design Research

We review the scholarly contributions that utilise Natural Language Processing (NLP) techniques to support the design process. Using a heuristic approach, we gathered 223 articles that are published in 32 journals within the period 1991-present. We present state-of-the-art NLP in-and-for design research by reviewing these articles according to the type of natural language text sources: internal reports, design concepts, discourse transcripts, technical publications, consumer opinions, and others. Upon summarizing and identifying the gaps in these contributions, we utilise an existing design in…

A Survey of the Usages of Deep Learning in Natural Language Processing

Over the last several years, the field of natural language processing has been propelled forward by an explosion in the use of deep learning models. This survey provides a brief introduction to the field and a quick overview of deep learning architectures and methods. It then sifts through the plethora of recent studies and summarizes a large assortment of relevant contributions. Analyzed research areas include several core linguistic processing issues in addition to a number of applications of computational linguistics. A discussion of the current state of the art is then provided along with …

Challenges Encountered in Turkish Natural Language Processing Studies

Natural language processing is a branch of computer science that combines artificial intelligence with linguistics. It aims to analyze a language element such as writing or speaking with software and convert it into information. Considering that each language has its own grammatical rules and vocabulary diversity, the complexity of the studies in this field is somewhat understandable. For instance, Turkish is a very interesting language in many ways. Examples of this are agglutinative word structure, consonant/vowel harmony, a large number of productive derivational morphemes (practically infi…

Spoiler Alert: Using Natural Language Processing to Detect Spoilers in Book Reviews

This paper presents an NLP (Natural Language Processing) approach to detecting spoilers in book reviews, using the University of California San Diego (UCSD) Goodreads Spoiler dataset. We explored the use of LSTM, BERT, and RoBERTa language models to perform spoiler detection at the sentence-level. This was contrasted with a UCSD paper which performed the same task, but using handcrafted features in its data preparation. Despite eschewing the use of handcrafted features, our results from the LSTM model were able to slightly exceed the UCSD team's performance in spoiler detection.

使用人工智能进行项目的现场测试：基于transformers的自然语言处理

Field-testing items using artificial intelligence: Natural language processing with transformers

本研究通过让RoBERTa模型的5000个变体完成29道多选题的英语读写考试，测试了项目的心理测量学属性，发现其与人类考生数据具有一定一致性。

Recent Trends in Deep Learning Based Natural Language Processing

Deep learning methods employ multiple processing layers to learn hierarchical representations of data and have produced state-of-the-art results in many domains. Recently, a variety of model designs and methods have blossomed in the context of natural language processing (NLP). In this paper, we review significant deep learning related models and methods that have been employed for numerous NLP tasks and provide a walk-through of their evolution. We also summarize, compare and contrast the various models and put forward a detailed understanding of the past, present and future of deep learning …

将“自然”放回自然语言处理中

Putting Natural in Natural Language Processing

本文指出NLP过度依赖文本而忽视口语，主张借助深度学习的进展统一语音处理与主流NLP，将口语视为人类交流的主要模式，从而构建更高效、拟人且超越文本模态的系统。

Doing Natural Language Processing in A Natural Way: An NLP toolkit based on object-oriented knowledge base and multi-level grammar base

We introduce an NLP toolkit based on object-oriented knowledge base and multi-level grammar base. This toolkit focuses on semantic parsing, it also has abilities to discover new knowledge and grammar automatically, new discovered knowledge and grammar will be identified by human, and will be used to update the knowledge base and grammar base. This process can be iterated many times to improve the toolkit continuously.

The Role of Explanatory Value in Natural Language Processing

A key aim of science is explanation, yet the idea of explaining language phenomena has taken a backseat in mainstream Natural Language Processing (NLP) and many other areas of Artificial Intelligence. I argue that explanation of linguistic behaviour should be a main goal of NLP, and that this is not the same as making NLP models explainable. To illustrate these ideas, some recent models of human language production are compared with each other. I conclude by asking what it would mean for NLP research and institutional policies if our community took explanatory value seriously, while heeding so…

Machine Learning Meets Natural Language Processing -- The story so far

Natural Language Processing (NLP) has evolved significantly over the last decade. This paper highlights the most important milestones of this period while trying to pinpoint the contribution of each individual model and algorithm to the overall progress. Furthermore, it focuses on issues still remaining to be solved, emphasizing the groundbreaking proposals of Transformers, BERT, and all the similar attention-based models.

HULK: An Energy Efficiency Benchmark Platform for Responsible Natural Language Processing

Computation-intensive pretrained models have been taking the lead of many natural language processing benchmarks such as GLUE. However, energy efficiency in the process of model training and inference becomes a critical bottleneck. We introduce HULK, a multi-task energy efficiency benchmarking platform for responsible natural language processing. With HULK, we compare pretrained models' energy efficiency from the perspectives of time and cost. Baseline benchmarking results are provided for further analysis. The fine-tuning efficiency of different pretrained models can differ a lot among differ…

Concept-Based Embeddings for Natural Language Processing

In this work, we focus on effectively leveraging and integrating information from concept-level as well as word-level via projecting concepts and words into a lower dimensional space while retaining most critical semantics. In a broad context of opinion understanding system, we investigate the use of the fused embedding for several core NLP tasks: named entity detection and classification, automatic speech recognition reranking, and targeted sentiment analysis.

自然语言处理系统与语言模态的差距分析

Gap Analysis of Natural Language Processing Systems with respect to Linguistic Modality

情态是语言学中语法的重要组成部分。它让说话者表达对事态的态度、评价或潜力。它意味着不同的意义，因此根据上下文有不同的感知。本文介绍了一个帐户显示的差距，在功能的当前状态的最先进的自然语言处理（NLP）系统。研究了语言情态的语境性质。本文综述了自然语言处理系统处理情态的工作和逻辑方法。它将人类认知和智能视为多层次的方法，可以由智能学习系统实现。最后，讨论了该领域目前的研究进展，为今后的研究提供了参考。

The impact of lexical and grammatical processing on generating code from natural language

Considering the seq2seq architecture of TranX for natural language to code translation, we identify four key components of importance: grammatical constraints, lexical preprocessing, input representations, and copy mechanisms. To study the impact of these components, we use a state-of-the-art architecture that relies on BERT encoder and a grammar-based decoder for which a formalization is provided. The paper highlights the importance of the lexical substitution component in the current natural language to code systems.

自然语言处理：她需要一些旧的东西和一些新的东西（也许是借来的东西，也是蓝色的东西）

Natural language processing: she needs something old and something new (maybe something borrowed and something blue, too)

鉴于目前自然语言处理的工作状态，本文首先认为，科学和应用的进步需要重新关注语言是关于什么的，广义地说是关于世界的;第二，对总结任务的攻击，这是越来越重要的电子文本资源的增长，并需要了解的作用，大-尺度语篇结构在标注重要文本内容时，是一个很好的前进方向。

Understanding scholarly Natural Language Processing system diagrams through application of the Richards-Engelhardt framework

We utilise Richards-Engelhardt framework as a tool for understanding Natural Language Processing systems diagrams. Through four examples from scholarly proceedings, we find that the application of the framework to this ecological and complex domain is effective for reflecting on these diagrams. We argue for vocabulary to describe multiple-codings, semiotic variability, and inconsistency or misuse of visual encoding principles in diagrams. Further, for application to scholarly Natural Language Processing systems, and perhaps systems diagrams more broadly, we propose the addition of "Grouping by…

Word Embeddings for Chemical Patent Natural Language Processing

We evaluate chemical patent word embeddings against known biomedical embeddings and show that they outperform the latter extrinsically and intrinsically. We also show that using contextualized embeddings can induce predictive models of reasonable performance for this domain over a relatively small gold standard.

BNLP: Natural language processing toolkit for Bengali language

BNLP is an open source language processing toolkit for Bengali language consisting with tokenization, word embedding, POS tagging, NER tagging facilities. BNLP provides pre-trained model with high accuracy to do model based tokenization, embedding, POS tagging, NER tagging task for Bengali language. BNLP pre-trained model achieves significant results in Bengali text tokenization, word embedding, POS tagging and NER tagging task. BNLP is using widely in the Bengali research communities with 16K downloads, 119 stars and 31 forks. BNLP is available at https://github.com/sagorbrur/bnlp.

Syntactic Enhancement to VSIMM for Roadmap Based Anomalous Trajectory Detection: A Natural Language Processing Approach

The aim of syntactic tracking is to classify spatio-temporal patterns of a target's motion using natural language processing models. In this paper, we generalize earlier work by considering a constrained stochastic context free grammar (CSCFG) for modeling patterns confined to a roadmap. The constrained grammar facilitates modeling specific directions and road names in a roadmap. We present a novel particle filtering algorithm that exploits the CSCFG model for estimating the target's patterns. This meta-level algorithm operates in conjunction with a base-level tracking algorithm. Extensive num…

自然语言处理中CNN和RNN的比较研究

Comparative Study of CNN and RNN for Natural Language Processing

深度神经网络（DNN）彻底改变了自然语言处理（NLP）领域。卷积神经网络（CNN）和递归神经网络（RNN）是DNN架构的两种主要类型，被广泛探索以处理各种NLP任务。CNN被认为擅长于提取位置不变的特征，而RNN则擅长于按顺序提取建模单元。由于CNN和RNN之间的斗争，许多NLP任务的最新技术经常发生变化。这项工作是CNN和RNN在广泛的代表性NLP任务上的首次系统比较，旨在为DNN的选择提供基本指导。

Information Propagation by Composited Labels in Natural Language Processing

In natural language processing (NLP), labeling on regions of text, such as words, sentences and paragraphs, is a basic task. In this paper, label is defined as map between mention of entity in a region on text and context of entity in a broader region on text containing the mention. This definition naturally introduces linkage of entities induced from inclusion relation of regions, and connected entities form a graph representing information flow defined by map. It also enables calculation of information loss through map using entropy, and entropy lost is regarded as distance between two entit…

Data Readiness for Natural Language Processing

This document concerns data readiness in the context of machine learning and Natural Language Processing. It describes how an organization may proceed to identify, make available, validate, and prepare data to facilitate automated analysis methods. The contents of the document is based on the practical challenges and frequently asked questions we have encountered in our work as an applied research institute with helping organizations and companies, both in the public and private sectors, to use data in their business processes.

自然语言处理与多模态股票价格预测

Natural Language Processing and Multimodal Stock Price Prediction

本文提出利用股票百分比变化作为训练数据，并结合专门的BERT NLP模型与多模态数据分析来预测股票价格趋势，证明了小型模型在准确预测整体股票趋势方面的能力及特定数据特征的有效性。

Features matching using natural language processing

The feature matching is a basic step in matching different datasets. This article proposes shows a new hybrid model of a pretrained Natural Language Processing (NLP) based model called BERT used in parallel with a statistical model based on Jaccard similarity to measure the similarity between list of features from two different datasets. This reduces the time required to search for correlations or manually match each feature from one dataset to another.