In this paper, we present a transfer learning system to perform technical domain identification on multilingual text data. We have submitted two runs, one uses the transformer model BERT, and the other uses XLM-ROBERTa with the CNN model for text classification. These models allowed us to identify the domain of the given sentences for the ICON 2020 shared Task, TechDOfication: Technical Domain Identification. Our system ranked the best for the subtasks 1d, 1g for the given TechDOfication dataset.
Center for Language and Cognition（语言与认知中心）；University of Groningen（格罗宁根大学）
This technical report introduces an adapted version of the LOME frame semantic parsing model (Xia et al., EACL 2021) which is capable of automatically annotating texts according to the "Kicktionary" domain-specific framenet resource. Several methods for training a model even with limited available training data are proposed. While there are some challenges for evaluation related to the nature of the available annotations, preliminary results are very promising, with the best model reaching F1-scores of 0.83 (frame prediction) and 0.81 (semantic role prediction).
Arab Academy for Science, Technology, and Maritime Transport（阿拉伯科学、技术与海运学院）；College of Computing & Information Technology（计算与信息技术学院）；Columbus State University（哥伦布州立大学）；TSYS School of Computer Science（TSYS计算机学院）
This thesis presents a language-independent text classification model by introduced two new encoding methods "BUNOW" and "BUNOC" used for feeding the raw text data into a new CNN spatial architecture with vertical and horizontal convolutional process instead of commonly used methods like one hot vector or word representation (i.e. word2vec) with temporal CNN architecture. The proposed model can be classified as hybrid word-character model in its work methodology because it consumes less memory space by using a fewer neural network parameters as in character level representation, in addition to…
Insight Centre for Data Analytics（数据洞察中心）；National University of Ireland, Galway（爱尔兰国立大学戈尔韦分校）；Aylien Ltd.（Aylien有限公司）
This paper describes our deep learning-based approach to multilingual aspect-based sentiment analysis as part of SemEval 2016 Task 5. We use a convolutional neural network (CNN) for both aspect extraction and aspect-based sentiment analysis. We cast aspect extraction as a multi-label classification problem, outputting probabilities over aspects parameterized by a threshold. To determine the sentiment towards an aspect, we concatenate an aspect vector with every word embedding and apply a convolution over it. Our constrained system (unconstrained for English) achieves competitive results across…
This paper describes the UMDSub system that participated in Task 2 of SemEval-2018. We developed a system that predicts an emoji given the raw text in a English tweet. The system is a Multi-channel Convolutional Neural Network based on subword embeddings for the representation of tweets. This model improves on character or word based methods by about 2\%. Our system placed 21st of 48 participating systems in the official evaluation.
Modal sense classification (MSC) is a special WSD task that depends on the meaning of the proposition in the modal's scope. We explore a CNN architecture for classifying modal sense in English and German. We show that CNNs are superior to manually designed feature-based classifiers and a standard NN classifier. We analyze the feature maps learned by the CNN and identify known and previously unattested linguistic features. We benchmark the CNN on a standard WSD task, where it compares favorably to models using sense-disambiguated target vectors.
University of Helsinki（赫尔辛基大学）；Rootroo Ltd（Rootroo有限公司）
The study forms a technical report of various tasks that have been performed on the materials collected and published by Finnish ethnographer and linguist, Matthias Alexander Castrén (1813-1852). The Finno-Ugrian Society is publishing Castrén's manuscripts as new critical and digital editions, and at the same time different research groups have also paid attention to these materials. We discuss the workflows and technical infrastructure used, and consider how datasets that benefit different computational tasks could be created to further improve the usability of these materials, and also to ai…
Samsung AI Centre（三星人工智能中心）；Peak AI；University of Edinburgh（爱丁堡大学）
Semantic parses are directed acyclic graphs (DAGs), so semantic parsing should be modeled as graph prediction. But predicting graphs presents difficult technical challenges, so it is simpler and more common to predict the linearized graphs found in semantic parsing datasets using well-understood sequence models. The cost of this simplicity is that the predicted strings may not be well-formed graphs. We present recurrent neural network DAG grammars, a graph-aware sequence model that ensures only well-formed graphs while sidestepping many difficulties in graph prediction. We test our model on th…
New York University（纽约大学）；IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）
Convolutional neural networks (CNNs) are a standard component of many current state-of-the-art Large Vocabulary Continuous Speech Recognition (LVCSR) systems. However, CNNs in LVCSR have not kept pace with recent advances in other domains where deeper neural networks provide superior performance. In this paper we propose a number of architectural advances in CNNs for LVCSR. First, we introduce a very deep convolutional network architecture with up to 14 weight layers. There are multiple convolutional layers before each pooling layer, with small 3x3 kernels, inspired by the VGG Imagenet 2014 ar…
The pre-trained language models have achieved great successes in various natural language understanding (NLU) tasks due to its capacity to capture the deep contextualized information in text by pre-training on large-scale corpora. One of the fundamental components in pre-trained language models is the vocabulary, especially for training multilingual models on many different languages. In the technical report, we present our practices on training multilingual pre-trained language models with BBPE: Byte-Level BPE (i.e., Byte Pair Encoding). In the experiment, we adopted the architecture of NEZHA…
Nanjing University of Science and Technology（南京理工大学）；JD AI Research（京东人工智能研究院）
Image paragraph generation is the task of producing a coherent story (usually a paragraph) that describes the visual content of an image. The problem nevertheless is not trivial especially when there are multiple descriptive and diverse gists to be considered for paragraph generation, which often happens in real images. A valid question is how to encapsulate such gists/topics that are worthy of mention from an image, and then describe the image from one topic to another but holistically with a coherent structure. In this paper, we present a new design --- Convolutional Auto-Encoding (CAE) that…
Graph Convolutional Networks (GCN) have achieved state-of-art results on single text classification tasks like sentiment analysis, emotion detection, etc. However, the performance is achieved by testing and reporting on resource-rich languages like English. Applying GCN for multi-task text classification is an unexplored area. Moreover, training a GCN or adopting an English GCN for Indian languages is often limited by data availability, rich morphological variation, syntax, and semantic differences. In this paper, we study the use of GCN for the Telugu language in single and multi-task setting…
University of Bucharest（布加勒斯特大学）；University Politehnica of Bucharest（布加勒斯特理工大学）
Current benchmark tasks for natural language processing contain text that is qualitatively different from the text used in informal day to day digital communication. This discrepancy has led to severe performance degradation of state-of-the-art NLP models when fine-tuned on real-world data. One way to resolve this issue is through lexical normalization, which is the process of transforming non-standard text, usually from social media, into a more standardized form. In this work, we propose a sentence-level sequence-to-sequence model based on mBART, which frames the problem as a machine transla…
Institute of Artificial Intelligence (TeleAI), China Telecom Corp Ltd（中国电信集团有限公司人工智能研究院（TeleAI））；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
Amazon Inc.（亚马逊公司）；Salesforce Research（赛富时研究院）；University of Pittsburgh（匹兹堡大学）；Chinese University of Hong Kong（香港中文大学）
Keyphrase generation is the task of automatically predicting keyphrases given a piece of long text. Despite its recent flourishing, keyphrase generation on non-English languages haven't been vastly investigated. In this paper, we call attention to a new setting named multilingual keyphrase generation and we contribute two new datasets, EcommerceMKP and AcademicMKP, covering six languages. Technically, we propose a retrieval-augmented method for multilingual keyphrase generation to mitigate the data shortage problem in non-English languages. The retrieval-augmented model leverages keyphrase ann…
Charles University, Faculty of Mathematics and Physics（查理大学数学与物理学院）
We introduce an approach to multilingual speech synthesis which uses the meta-learning concept of contextual parameter generation and produces natural-sounding multilingual speech using more languages and less training data than previous approaches. Our model is based on Tacotron 2 with a fully convolutional input text encoder whose weights are predicted by a separate parameter generator network. To boost voice cloning, the model uses an adversarial speaker classifier with a gradient reversal layer that removes speaker-specific information from the encoder. We arranged two experiments to compa…
Instituto Superior Técnico & Universidade de Lisboa (Lisbon ELLIS Unit)（里斯本ELLIS单位（理工学院与里斯本大学））；Instituto de Telecomunicações（电信研究院）；MICS, CentraleSupélec, Université Paris-Saclay（MICS、中央圣艾尔布兰理工大学、巴黎萨克雷大学）；Acolad；Carnegie Mellon University（卡内基梅隆大学）；University of Edinburgh（爱丁堡大学）；Diabolocom；Aveni；OutSystems；Sword Health；Sorbonne Université, CNRS, ISIR（索邦大学、国家科学研究中心、ISIR）；Artefact Research Center（Artefact研究机构）；TransPerfect
This technical report details several improvements to the visual concept detector banks built on images from the Multilingual Visual Sentiment Ontology (MVSO). The detector banks are trained to detect a total of 9,918 sentiment-biased visual concepts from six major languages: English, Spanish, Italian, French, German and Chinese. In the original MVSO release, adjective-noun pair (ANP) detectors were trained for the six languages using an AlexNet-styled architecture by fine-tuning from DeepSentiBank. Here, through a more extensive set of experiments, parameter tuning, and training runs, we deta…
