Large generative language models have been very successful for English, but other languages lag behind, in part due to data and computational limitations. We propose a method that may overcome these problems by adapting existing pre-trained models to new languages. Specifically, we describe the adaptation of English GPT-2 to Italian and Dutch by retraining lexical embeddings without tuning the Transformer layers. As a result, we obtain lexical embeddings for Italian and Dutch that are aligned with the original English lexical embeddings. Additionally, we scale up complexity by transforming relearned lexical embeddings of GPT-2 small to the GPT-2 medium embedding space. This method minimises the amount of training and prevents losing information during adaptation that was learned by GPT-2. English GPT-2 models with relearned lexical embeddings can generate realistic sentences in Italian and Dutch. Though on average these sentences are still identifiable as artificial by humans, they are assessed on par with sentences generated by a GPT-2 model fully trained from scratch.
Levin et al. (2019) show experimentally that the interpretations of novel English noun compounds (e.g., stew skillet), while not fully compositional, are highly predictable based on whether the modifier and head refer to artifacts or natural kinds. Is the large language model GPT-3 governed by the same interpretive principles? To address this question, we first compare Levin et al.'s experimental data with GPT-3 generations, finding a high degree of similarity. However, this evidence is consistent with GPT3 reasoning only about specific lexical items rather than the more abstract conceptual ca…
University of California, Santa Barbara（加州大学圣巴巴拉分校）
The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts. However, NLP models have historically been developed using dominant English varieties, such as Standard American English (SAE), due to text corpora availability. We investigate the performance of GPT-2 on AAVE text by creating a dataset of intent-equivalent parallel AAVE/SAE tweet pairs, thereby isolating syntactic structure and AAVE- or SAE-specific language for each pair. We evaluate each sample and its GPT-2 generated text with pre…
Optimising ChatGPT for creativity in literary translation: A case study from English into Dutch, Chinese, Catalan and Spanish
Centre for Language and Cognition, University of Groningen（语言与认知中心，格罗宁根大学）；Departament de Llenguatges i Sistemes Informàtics, Universitat d’Alacant（语言与信息系统部门，阿尔阿罕特大学）；Departament de Traducció i Comunicació, Universitat Jaume I（翻译与传播部门， Jaime I 大学）
Indirect Question Answering in English, German and Bavarian: A Challenging Task for High- and Low-Resource Languages Alike
MaiNLP lab, Center for Information and Language Processing, LMU Munich, Germany（MaiNLP实验室，信息与语言处理中心，慕尼黑大学，德国）；Munich Center for Machine Learning, Germany（慕尼黑机器学习中心，德国）
Lightweight but Linguistically Fair? Quantisation Robustness across Bavarian, German, and English
Leibniz Supercomputing Centre (LRZ)（莱布尼茨超级计算中心）；Bavarian Academy of Sciences and Humanities（巴伐利亚科学与人文学院）；TUM University Hospital（慕尼黑工业大学医院）；Technical University of Munich（慕尼黑工业大学）
While specifications for digital systems are provided in natural language, engineers undertake significant efforts to translate them into the programming languages understood by compilers for digital systems. Automating this process allows designers to work with the language in which they are most comfortable --the original natural language -- and focus instead on other downstream design challenges. We explore the use of state-of-the-art machine learning (ML) to automatically derive Verilog snippets from English via fine-tuning GPT-2, a natural language ML system. We describe our approach for …
Generative Pre-trained Transformer (GPT) models have shown remarkable capabilities for natural language generation, but their performance for machine translation has not been thoroughly investigated. In this paper, we present a comprehensive evaluation of GPT models for machine translation, covering various aspects such as quality of different GPT models in comparison with state-of-the-art research and commercial systems, effect of prompting strategies, robustness towards domain shifts and document-level translation. We experiment with eighteen different translation directions involving high a…
Generative Pre-trained Transformers (GPTs) have recently been scaled to unprecedented sizes in the history of machine learning. These models, solely trained on the language modeling objective, have been shown to exhibit outstanding few-shot learning capabilities in a number of different tasks. Nevertheless, aside from anecdotal experiences, little is known regarding their multilingual capabilities, given the fact that the pre-training corpus is almost entirely composed of English text. In this work, we investigate the multilingual skills of GPT-3, focusing on one language that barely appears i…
Rapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese
Large Language Model-Driven Dynamic Assessment of Grammatical Accuracy in English Language Learner Writing
National Institute of Standards（国家标准 institute）；Department of Physics, Colorado State University（物理系，科罗拉多州立大学）；Electrical Engineering Department, University of Colorado（电气工程系，科罗拉多大学）
Obtaining text datasets with semantic annotations is an effortful process, yet crucial for supervised training in natural language processsing (NLP). In general, developing and applying new NLP pipelines in domain-specific contexts for tasks often requires custom designed datasets to address NLP tasks in supervised machine learning fashion. When operating in non-English languages for medical data processing, this exposes several minor and major, interconnected problems such as lack of task-matching datasets as well as task-specific pre-trained models. In our work we suggest to leverage pretrai…
Benchmarking GPT-4 against Human Translators: A Comprehensive Evaluation Across Languages, Domains, and Expertise Levels
Zhejiang University（浙江大学）；School of Engineering, Westlake University（西湖大学工程学院）；University College London（伦敦大学学院）；University of Cambridge（剑桥大学）；LanBridge Group（兰桥集团）；Institute of Advanced Technology, Westlake Institute for Advanced Study（西湖高等研究院先进技术研究院）
AI4Math: A Native Spanish Benchmark for University-Level Mathematical Reasoning in Large Language Models
Centro de Investigación Científica y de Educación Superior de Ensenada（恩森亚达科学与高等教育研究中心）；Facultad de Ciencias, UNAM（科学学院，国立自治大学）；Facultad de Matemáticas, Universidad Veracruzana（数学学院，韦拉克鲁斯大学）
