It is reasonable to hypothesize that the divergence patterns formulated by historical linguists and typologists reflect constraints on human languages, and are thus consistent with Second Language Acquisition (SLA) in a certain way. In this paper, we validate this hypothesis on ten Indo-European languages. We formalize the delexicalized transfer as interpretable tree-to-string and tree-to-tree patterns which can be automatically induced from web data by applying neural syntactic parsing and grammar induction technologies. This allows us to quantitatively probe cross-linguistic transfer and extend inquiries of SLA. We extend existing works which utilize mixed features and support the agreement between delexicalized cross-linguistic transfer and the phylogenetic structure resulting from the historical-comparative paradigm.
University of Oslo（奥斯陆大学）；Department of Informatics（信息学系）
The age of the root of the Indo-European language family has received much attention since the application of Bayesian phylogenetic methods by Gray and Atkinson(2003). The root age of the Indo-European family has tended to decrease from an age that supported the Anatolian origin hypothesis to an age that supports the Steppe origin hypothesis with the application of new models (Chang et al., 2015). However, none of the published work in the Indo-European phylogenetics studied the effect of tree priors on phylogenetic analyses of the Indo-European family. In this paper, I intend to fill this gap…
Using Phylogenetic Algebraic Geometry, we analyze computationally the phylogenetic tree of subfamilies of the Indo-European language family, using data of syntactic structures. The two main sources of syntactic data are the SSWL database and Longobardi's recent data of syntactic parameters. We compute phylogenetic invariants and likelihood functions for two sets of Germanic languages, a set of Romance languages, a set of Slavic languages and a set of early Indo-European languages, and we compare the results with what is known through historical linguistics.
This paper describes Charles University submission for Multilingual Low-Resource Translation for Indo-European Languages shared task at WMT21. We competed in translation from Catalan into Romanian, Italian and Occitan. Our systems are based on shared multilingual model. We show that using joint model for multiple similar language pairs improves upon translation quality in each pair. We also demonstrate that chararacter-level bilingual models are competitive for very similar language pairs (Catalan-Occitan) but less so for more distant pairs. We also describe our experiments with multi-task lea…
Natural Language Processing Lab（自然语言处理实验室）；The University of British Columbia（英属哥伦比亚大学）
In this work, we investigate methods for the challenging task of translating between low-resource language pairs that exhibit some level of similarity. In particular, we consider the utility of transfer learning for translating between several Indo-European low-resource languages from the Germanic and Romance language families. In particular, we build two main classes of transfer-based systems to study how relatedness can benefit the translation performance. The primary system fine-tunes a model pre-trained on a related language pair and the contrastive system fine-tunes one pre-trained on an …
Quantitative Measurement of Language Distance among Closely Related Indo-European Languages Using Pretrained Language Models: A Case Study on the North Germanic Branch
Facultad de Ciencias, Universidad Nacional Autónoma de México（墨西哥国立自治大学理学院）；Instituto de Física, Universidad Nacional Autónoma de México（墨西哥国立自治大学物理研究所）；Instituto de Investigaciones en Matemáticas Aplicadas y Sistemas, Universidad Nacional Autónoma de México（墨西哥国立自治大学应用数学与系统研究所）；Centro de Ciencias de la Complejidad, Universidad Nacional Autónoma de México（墨西哥国立自治大学复杂性科学中心）；Santa Fe Institute（圣塔菲研究所）
A recent increase in data availability has allowed the possibility to perform different statistical linguistic studies. Here we use the Google Books Ngram dataset to analyze word flow among English, French, German, Italian, and Spanish. We study what we define as ``migrant words'', a type of loanwords that do not change their spelling. We quantify migrant words from one language to another for different decades, and notice that most migrant words can be aggregated in semantic fields and associated to historic events. We also study the statistical properties of accumulated migrant words and the…
We study the persistent homology of the data set of syntactic parameters of the world languages. We show that, while homology generators behave erratically over the whole data set, non-trivial persistent homology appears when one restricts to specific language families. Different families exhibit different persistent homology. We focus on the cases of the Indo-European and the Niger-Congo families, for which we compare persistent homology over different cluster filtering values. We investigate the possible significance, in historical linguistic terms, of the presence of persistent generators o…
A simple machine learning model of pluralisation as a linear regression problem minimising a p-adic metric substantially outperforms even the most robust of Euclidean-space regressors on languages in the Indo-European, Austronesian, Trans New-Guinea, Sino-Tibetan, Nilo-Saharan, Oto-Meanguean and Atlantic-Congo language families. There is insufficient evidence to support modelling distinct noun declensions as a p-adic neighbourhood even in Indo-European languages.
While most of the speech and natural language systems which were developed for English and other Indo-European languages neglect the morphological processing and integrate speech and natural language at the word level, for the agglutinative languages such as Korean and Japanese, the morphological processing plays a major role in the language processing since these languages have very complex morphological phenomena and relatively simple syntactic functionality. Obviously degenerated morphological processing limits the usable vocabulary size for the system and word-level dictionary results in e…
Institute of Neuroinformatics, University of Zürich and ETH Zürich（苏黎世大学与苏黎世联邦理工学院神经信息学研究所）
Character-level Neural Machine Translation (NMT) models have recently achieved impressive results on many language pairs. They mainly do well for Indo-European language pairs, where the languages share the same writing system. However, for translating between Chinese and English, the gap between the two different writing systems poses a major challenge because of a lack of systematic correspondence between the individual linguistic units. In this paper, we enable character-level NMT for Chinese, by breaking down Chinese characters into linguistic units similar to that of Indo-European language…
University of Hildesheim（希尔德斯海姆大学）；LDRP-ITR；University of Duisburg-Essen（杜伊斯堡-埃森大学）；University of Bedfordshire（贝德福德大学）；University of Leeds（利兹大学）；University of Bamberg（班贝格大学）；Dalhousie University（达尔豪斯大学）；DA-IICT
With the growth of social media, the spread of hate speech is also increasing rapidly. Social media are widely used in many countries. Also Hate Speech is spreading in these countries. This brings a need for multilingual Hate Speech detection algorithms. Much research in this area is dedicated to English at the moment. The HASOC track intends to provide a platform to develop and optimize Hate Speech detection algorithms for Hindi, German and English. The dataset is collected from a Twitter archive and pre-classified by a machine learning system. HASOC has two sub-task for all three languages: …
The rapid growth in data on the internet requires a data mining process to reach a decision to support insight. The Persian language has strong potential for deep research in any aspect of natural language processing, especially sentimental analysis approach. Thousands of websites and blogs updates and modifies by Persian users around the world that contains millions of Persian context. This range of application requires a comprehensive structured framework to extract beneficial information for helping enterprises to enhance their business and initiate a customer-centric management process by …
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；MMT Srl（MMT有限公司）
Both research and commercial machine translation have so far neglected the importance of properly handling the spelling, lexical and grammar divergences occurring among language varieties. Notable cases are standard national varieties such as Brazilian and European Portuguese, and Canadian and European French, which popular online machine translation services are not keeping distinct. We show that an evident side effect of modeling such varieties as unique classes is the generation of inconsistent translations. In this work, we investigate the problem of training neural machine translation fro…
How do neural networks "perceive" speech sounds from unknown languages? Does the typological similarity between the model's training language (L1) and an unknown language (L2) have an impact on the model representations of L2 speech signals? To answer these questions, we present a novel experimental design based on representational similarity analysis (RSA) to analyze acoustic word embeddings (AWEs) -- vector representations of variable-duration spoken-word segments. First, we train monolingual AWE models on seven Indo-European languages with various degrees of typological similarity. We then …
We present a pattern matching method for compiling a bilingual lexicon of nouns and proper nouns from unaligned, noisy parallel texts of Asian/Indo-European language pairs. Tagging information of one language is used. Word frequency and position information for high and low frequency words are represented in two different vector forms for pattern matching. New anchor point finding and noise elimination techniques are introduced. We obtained a 73.1\% precision. We also show how the results can be used in the compilation of domain-specific noun phrases.
ENEA, Italian National Agency for New Technologies, Energy and Sustainable Economic Development（意大利国家新技术、能源与可持续经济发展署（ENEA））；Università di Roma “Sapienza”（罗马第一大学（Sapienza 罗马大学））；Centre for Statistical Mechanics and Complexity (SMC), CNR-INFM（意大利国家研究委员会-国家物质物理研究所 统计力学与复杂性中心（SMC））；INFN（意大利国家核物理研究所（INFN））；Università di Bari（巴里大学）
We classify twenty-one Indo-European languages starting from written text. We use neural networks in order to define a distance among different languages, construct a dendrogram and analyze the ultrametric structure that emerges. Four or five subgroups of languages are identified, according to the "cut" of the dendrogram, drawn with an entropic criterion. The results and the method are discussed.
On the Cross-lingual Transferability of Pre-trained wav2vec2-based Models
Department of Informatics（信息学院）；Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）；National School of Statistical Sciences（国家统计科学学校）；Brazilian Institute of Geography and Statistics（巴西地理与统计研究所）；Institute of Systems and Robotics（系统与机器人研究所）；University of Coimbra（科英布拉大学）
MultiLexNorm++: A Unified Benchmark and a Generative Model for Lexical Normalization for Asian Languages
School of Information Science and Technology (VISTEC)（信息科学与技术学院）；University of Information Technology, Ho Chi Minh City（胡志明市信息技术大学）；Vietnam National University, Ho Chi Minh City（越南国家大学胡志明市分校）；Graduate School of Science and Engineering, Ehime University（Ehime大学研究生院）；D3 Center, The University of Osaka（大阪大学D3中心）；Chung-Ang University（Chung-Ang大学）；IT University of Copenhagen（哥本哈根IT大学）
Languages evolve over time in a process in which reproduction, mutation and extinction are all possible, similar to what happens to living organisms. Using this similarity it is possible, in principle, to build family trees which show the degree of relatedness between languages. The method used by modern glottochronology, developed by Swadesh in the 1950s, measures distances from the percentage of words with a common historical origin. The weak point of this method is that subjective judgment plays a relevant role. Recently we proposed an automated method that avoids the subjectivity, whose re…
PES University（PES大学）；CMR Institute of Technology（CMR理工学院）
Traditionally linguists have organized languages of the world as language families modelled as trees. In this work we take a contrarian approach and question the tree-based model that is rather restrictive. For example, the affinity that Sanskrit independently has with languages across Indo-European languages is better illustrated using a network model. We can say the same about inter-relationship between languages in India, where the inter-relationships are better discovered than assumed. To enable such a discovery, in this paper we have made use of instance-based learning techniques to assig…
Shanghai Jiao Tong University（上海交通大学）；City University of Hong Kong（香港城市大学）；University of Pennsylvania（宾夕法尼亚大学）；Brigham Young University（杨百翰大学）；University of Washington（华盛顿大学）；Indiana University Bloomington（印第安纳大学伯明顿分校）
Towards cross-cultural study of folksong lyrics with machine translation
Charles University（查理大学）；University of Music and Performing Arts Graz（格拉茨音乐与表演艺术大学）；Graduate School of Culture Technology（文化技术研究生院）；KAIST（韩国科学技术院）；Utrecht University（乌得勒支大学）；Institute of Ethnology（民族学研究所）；Czech Academy of Sciences（捷克科学院）；Estonian Literary Museum（爱沙尼亚文学博物馆）；NAS of Ukraine（乌克兰国家科学院）
