Giovanni Di Gennaro, Amedeo Buonanno, Antonio Di Girolamo, Armando Ospedale, Francesco A. N. Palmieri, Gianfranco Fedele
备注：
Presented at the 2019 Italian Workshop on Neural Networks (WIRN'19) - June 2019
发表信息：
Progresses in Artificial Intelligence and Neural Systems. Smart Innovation, Systems and Technologies, vol 184. Springer, Singapore - First Online: July 2020
Word representation is fundamental in NLP tasks, because it is precisely from the coding of semantic closeness between words that it is possible to think of teaching a machine to understand text. Despite the spread of word embedding concepts, still few are the achievements in linguistic contexts other than English. In this work, analysing the semantic capacity of the Word2Vec algorithm, an embedding for the Italian language is produced. Parameter setting such as the number of epochs, the size of the context window and the number of negatively backpropagated samples is explored.
A dissemination workshop for introducing young Italian students to NLP
Aston University（阿斯顿大学）；University of Bologna（博洛尼亚大学）；University of Trento（特伦托大学）；University of Pisa（比萨大学）；University of Trieste（的里雅斯特大学）；University of Groningen（格罗宁根大学）
INAIL - Dipartimento Innovazioni Tecnologiche（意大利国家工伤保险局技术创新部）；Politecnico di Milano（米兰理工大学）
Extracting patterns and useful information from Natural Language datasets is a challenging task, especially when dealing with data written in a language different from English, like Italian. Machine and Deep Learning, together with Natural Language Processing (NLP) techniques have widely spread and improved lately, providing a plethora of useful methods to address both Supervised and Unsupervised problems on textual information. We propose RECKONition, a NLP-based system for Industrial Accidents at Work Prevention. RECKONition, which is meant to provide Natural Language Understanding, Clusteri…
Teaching NLP with Bracelets and Restaurant Menus: An Interactive Workshop for Italian Students
University of Trento（特伦托大学）；Aston University（阿斯顿大学）；University of Bologna（博洛尼亚大学）；University of Pisa（比萨大学）；University of Trieste（的里雅斯特大学）；University of Groningen（格罗宁根大学）
Instituto Politécnico Nacional（国家理工学院）；Instituto Caro y Cuervo（卡罗-奎尔沃研究所）
We present our systems and findings on unsupervised lexical semantic change for the Italian language in the DIACR-Ita shared-task at EVALITA 2020. The task is to determine whether a target word has evolved its meaning with time, only relying on raw-text from two time-specific datasets. We propose two models representing the target words across the periods to predict the changing words using threshold and voting schemes. Our first model solely relies on part-of-speech usage and an ensemble of distance measures. The second model uses word embedding representation to extract the neighbor's relati…
DART: A Structured Dataset of Regulatory Drug Documents in Italian for Clinical NLP
University of Naples Federico II, Department of Electrical Engineering（那不勒斯费德里科二世大学电气工程系）；Consorzio Interuniversitario Nazionale per l'Informatica (CINI) - ITEM National Lab, Complesso Universitario Monte S.Angelo, Naples, Italy（国家信息计算联合研究所（CINI）- ITEM国家实验室，那不勒斯意大利大学综合体）；Northwestern University, Department of Computer Science, McCormick School of Engineering（西北大学计算机科学系，麦科姆工程学院）
In this paper we present SABRINA (Sentiment Analysis: a Broad Resource for Italian Natural language Applications) a manually annotated prior polarity lexical resource for Italian natural language applications in the field of opinion mining and sentiment induction. The resource consists in two different sets, an Italian dictionary of more than 277.000 words tagged with their prior polarity value, and a set of polarity modifiers, containing more than 200 words, which can be used in combination with non neutral terms of the dictionary in order to induce the sentiment of Italian compound terms. To…
Word embeddings play a significant role in today's Natural Language Processing tasks and applications. While pre-trained models may be directly employed and integrated into existing pipelines, they are often fine-tuned to better fit with specific languages or domains. In this paper, we attempt to improve available embeddings in the uncovered niche of the Italian medical domain through the combination of Contrastive Learning (CL) and Knowledge Graph Embedding (KGE). The main objective is to improve the accuracy of semantic similarity between medical terms, which is also used as an evaluation ta…
Bocconi University（博科尼大学）；Politecnico di Torino（都灵理工大学）；University of Milano-Bicocca（米兰比可卡大学）；University of Groningen（格罗宁根大学）
CLIP (Contrastive Language-Image Pre-training) is a very recent multi-modal model that jointly learns representations of images and texts. The model is trained on a massive amount of English data and shows impressive performance on zero-shot classification tasks. Training the same model on a different language is not trivial, since data in other languages might be not enough and the model needs high-quality translations of the texts to guarantee a good performance. In this paper, we present the first CLIP model for the Italian Language (CLIP-Italian), trained on more than 1.4 million image-tex…
Some languages allow arguments to be omitted in certain contexts. Yet human language comprehenders reliably infer the intended referents of these zero pronouns, in part because they construct expectations about which referents are more likely. We ask whether Neural Language Models also extract the same expectations. We test whether 12 contemporary language models display expectations that reflect human behavior when exposed to sentences with zero pronouns from five behavioral experiments conducted in Italian by Carminati (2005). We find that three models - XGLM 2.9B, 4.5B, and 7.5B - capture t…
This paper presents BAMBI (BAby language Models Boostrapped for Italian), a series of Baby Language Models (BabyLMs) trained on data that mimic the linguistic input received by a five-year-old Italian-speaking child. The BAMBI models are tested using a benchmark specifically designed to evaluate language models, which takes into account the amount of training input the models received. The BAMBI models are compared against a large language model (LLM) and a multimodal language model (VLM) to study the contribution of extralinguistic information for language acquisition. The results of our eval…
University of Pisa（比萨大学）；University of Groningen（格罗宁根大学）；Istituto di Linguistica Computazionale “Antonio Zampolli”（“安东尼奥·赞波利”计算语言学研究所）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
In the last few years, pre-trained neural architectures have provided impressive improvements across several NLP tasks. Still, generative language models are available mainly for English. We develop GePpeTto, the first generative language model for Italian, built using the GPT-2 architecture. We provide a thorough analysis of GePpeTto's quality by means of both an automatic and a human-based evaluation. The automatic assessment consists in (i) calculating perplexity across different genres and (ii) a profiling analysis over GePpeTto's writing characteristics. We find that GePpeTto's production…
DINFO, University of Florence（佛罗伦萨大学DINFO学院）；DIISM, University of Siena（锡耶纳大学DIISM学院）
Italian is a Romance language that has its roots in Vulgar Latin. The birth of the modern Italian started in Tuscany around the 14th century, and it is mainly attributed to the works of Dante Alighieri, Francesco Petrarca and Giovanni Boccaccio, who are among the most acclaimed authors of the medieval age in Tuscany. However, Italy has been characterized by a high variety of dialects, which are often loosely related to each other, due to the past fragmentation of the territory. Italian has absorbed influences from many of these dialects, as also from other languages due to dominion of portions…
Detecting Historical Turning Points in Italian Media: A Complex Systems Approach to a Diachronic News Corpus
Department of Physics and Chemistry, University of Palermo（帕尔米洛大学物理与化学系）；Institute for Cross-Disciplinary Physics and Complex Systems (IFISC), UIB-CSIC（跨学科物理与复杂系统研究所（IFISC）， UIB-CSIC）
University of Notre Dame（圣母大学）；University of Ioannina（约阿尼纳大学）；ICREA（加泰罗尼亚高等研究院）；Universitat Pompeu Fabra（庞培法布拉大学）
Most work on part-of-speech (POS) tagging is focused on high resource languages, or examines low-resource and active learning settings through simulated studies. We evaluate POS tagging techniques on an actual endangered language, Griko. We present a resource that contains 114 narratives in Griko, along with sentence-level translations in Italian, and provides gold annotations for the test set. Based on a previously collected small corpus, we investigate several traditional methods, as well as methods that take advantage of monolingual data or project cross-lingual POS tags. We show that the c…
Language Proficiency and F0 Entrainment: A Study of L2 English Imitation in Italian, French, and Slovak Speakers
Italian Institute of Technology（意大利技术研究院）；University of Ferrara（费拉拉大学）；Constantine the Philosopher University in Nitra（尼特拉君士坦丁哲学家大学）；Institute of Informatics, SAS（斯洛伐克科学院信息学研究所）
Modeling law search and retrieval as prediction problems has recently emerged as a predominant approach in law intelligence. Focusing on the law article retrieval task, we present a deep learning framework named LamBERTa, which is designed for civil-law codes, and specifically trained on the Italian civil code. To our knowledge, this is the first study proposing an advanced approach to law article prediction for the Italian legal system based on a BERT (Bidirectional Encoder Representations from Transformers) learning framework, which has recently attracted increased attention among deep learn…
In this work we analyze the performances of two of the most used word embeddings algorithms, skip-gram and continuous bag of words on Italian language. These algorithms have many hyper-parameter that have to be carefully tuned in order to obtain accurate word representation in vectorial space. We provide an accurate analysis and an evaluation, showing what are the best configuration of parameters for specific tasks.
Gender-Neutral Rewriting in Italian: Models, Approaches, and Trade-offs
University of Trento（特伦托大学）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核子研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（斯德哥尔摩大学）
Shiraz University（设拉子大学）；University of Calabria（卡拉布里亚大学）；Complutense University（马德里康普顿斯大学）
The effectiveness of a statistical machine translation system (SMT) is very dependent upon the amount of parallel corpus used in the training phase. For low-resource language pairs there are not enough parallel corpora to build an accurate SMT. In this paper, a novel approach is presented to extract bilingual Persian-Italian parallel sentences from a non-parallel (comparable) corpus. In this study, English is used as the pivot language to compute the matching scores between source and target sentences and candidate selection phase. Additionally, a new monolingual sentence similarity metric, No…
Laboratoire d'Informatique de Grenoble, Univ. Grenoble Alpes (UGA)（格勒诺布尔阿尔卑斯大学格勒诺布尔信息学实验室）；Univ. of Notre Dame（圣母大学）；Univ. of Ioannina（约阿尼纳大学）；UFRGS（南里奥格兰德联邦大学）；University of Essex（埃塞克斯大学）
This paper presents an extension to a very low-resource parallel corpus collected in an endangered language, Griko, making it useful for computational research. The corpus consists of 330 utterances (about 20 minutes of speech) which have been transcribed and translated in Italian, with annotations for word-level speech-to-transcription and speech-to-translation alignments. The corpus also includes morphosyntactic tags and word-level glosses. Applying an automatic unit discovery method, pseudo-phones were also generated. We detail how the corpus was collected, cleaned and processed, and we ill…
Natural Language Processing (NLP) is increasingly used as a key ingredient in critical decision-making systems such as resume parsers used in sorting a list of job candidates. NLP systems often ingest large corpora of human text, attempting to learn from past human behavior and decisions in order to produce systems that will make recommendations about our future world. Over 7000 human languages are being spoken today and the typical NLP pipeline underrepresents speakers of most of them while amplifying the voices of speakers of other languages. In this paper, a team including speakers of 8 lan…
Accurate detection of hate speech against politicians, policy making and political ideas is crucial to maintain democracy and free speech. Unfortunately, the amount of labelled data necessary for training models to detect hate speech are limited and domain-dependent. In this paper, we address the issue of classification of hate speech against policy makers from Twitter in Italian, producing the first resource of this type in this language. We collected and annotated 1264 tweets, examined the cases of disagreements between annotators, and performed in-domain and cross-domain hate speech classif…
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
Recently, state-of-the-art NLP models gained an increasing syntactic and semantic understanding of language, and explanation methods are crucial to understand their decisions. Occlusion is a well established method that provides explanations on discrete language data, e.g. by removing a language unit from an input and measuring the impact on a model's decision. We argue that current occlusion-based methods often produce invalid or syntactically incorrect language data, neglecting the improved abilities of recent NLP models. Furthermore, gradient-based explanation methods disregard the discrete…
AI "News" Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian
Istituto di Scienza e Tecnologia dell’Informazione “A. Faedo”（A. Faedo信息科学与技术研究所）；IT University of Copenhagen（哥本哈根信息技术大学）；Istituto di Linguistica Computazionale “Antonio Zampolli”（Antonio Zampolli计算语言学研究所）
