Lexical Semantic Change (LSC) is the phenomenon in which the meaning of a word change over time. Most studies on LSC focus on improving the performance of estimating the degree of LSC, however, it is often difficult to interpret how the meaning of a word change. Enhancing the interpretability of LSC is a significant challenge as it could lead to novel insights in this field. To tackle this challenge, we propose a method to map the semantic space of contextualized embeddings of words obtained by a pre-trained language model to a neurobiological feature space. In the neurobiological feature space, each dimension corresponds to a primitive feature of words, and its value represents the intensity of that feature. This enables humans to interpret LSC systematically. When employed for the estimation of the degree of LSC, our method demonstrates superior performance in comparison to the majority of the previous methods. In addition, given the high interpretability of the proposed method, several analyses on LSC are carried out. The results demonstrate that our method not only discovers interesting types of LSC that have been overlooked in previous studies but also effectively searches for words with specific types of LSC.
Mechanistic Diagnostics of Spatial Lexical Bias in Multimodal Large Language Model Spatial Reasoning
Kyoto University（京都大学）；NII LLMC（日本国立信息与通信技术研究所语言模型中心）；RIKEN AIP（日本理化学研究所先进理工研究所）；Case Western Reserve University（凯斯西储大学）；The Hong Kong Polytechnic University（香港理工大学）；The University of Osaka（大阪大学）；University of Tokyo（东京大学）
Bosch Center for Artificial Intelligence（博世人工智能中心）；Carnegie Mellon University（卡内基梅隆大学）
We study the utility of the lexical translation model (IBM Model 1) for English text retrieval, in particular, its neural variants that are trained end-to-end. We use the neural Model1 as an aggregator layer applied to context-free or contextualized query/document embeddings. This new approach to design a neural ranking system has benefits for effectiveness, efficiency, and interpretability. Specifically, we show that adding an interpretable neural Model 1 layer on top of BERT-based contextualized embeddings (1) does not decrease accuracy and/or efficiency; and (2) may overcome the limitation …
Word embeddings are substantially successful in capturing semantic relations among words. However, these lexical semantics are difficult to be interpreted. Definition modeling provides a more intuitive way to evaluate embeddings by utilizing them to generate natural language definitions of corresponding words. This task is of great significance for practical application and in-depth understanding of word representations. We propose a novel framework for definition modeling, which can generate reasonable and understandable context-dependent definitions. Moreover, we introduce usage modeling and…
ASELSAN Research Center（阿瑟兰研究中心）；Bilkent University（比尔肯特大学）；Ludwig Maximillian University（路德维希·马克西米利安大学）
As an ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related to a predefined concept to ta…
Multilingual Lexical Feature Analysis of Spoken Language for Predicting Major Depression Symptom Severity
University of Sussex（萨塞克斯大学）；Sant Joan de Déu Research Institut（圣琼德德乌研究所）；Mental Health Networking Biomedical Research Centre (CIBERSAM)（精神健康网络生物医学研究中心（CIBERSAM））；Amsterdam University Medical Centre, Vrije Universiteit（阿姆斯特丹自由大学阿姆斯特丹大学医学中心）；South London and Maudsley NHS Foundation Trust（南伦敦和莫兹利国民保健服务基金会信托）；University of Zaragoza（萨拉戈萨大学）；Biomedical Research Networking Centre in Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN)（生物工程、生物材料与纳米医学生物医学研究网络中心（CIBER-BBN））；Janssen Research and Development LLC（杨森研发有限责任公司）；University College London（伦敦大学学院）；Davos Alzheimer's Collaborative（达沃斯阿尔茨海默病协作组织）
Despite the increasing popularity of NLP in the humanities and social sciences, advances in model performance and complexity have been accompanied by concerns about interpretability and explanatory power for sociocultural analysis. One popular model that balances complexity and legibility is Word Mover's Distance (WMD). Ostensibly adapted for its interpretability, WMD has nonetheless been used and further developed in ways which frequently discard its most interpretable aspect: namely, the word-level distances required for translating a set of words into another set of words. To address this a…
How Does a Deep Neural Network Look at Lexical Stress in English Words?
Faculty of Electrical and Computer Engineering, Technion -- Israel Institute of Technology, Haifa, 3200003, Israel（电气与计算机工程学院，技术Ion -- 以色列理工学院，海法，3200003，以色列）；Department of Linguistics, Northwestern University, Evanston, Illinois 60208, USA（语言学系，西北大学，埃文斯顿，伊利诺伊州60208，美国）
A Methodology for Explainable Large Language Models with Integrated Gradients and Linguistic Analysis in Text Classification
Bioinformatics Multidisciplinary Environment (BioME)（生物信息学多学科环境（BioME））；Digital Metropolis Institute (IMD)（数字大都市研究所（IMD））；Federal University of Rio Grande do Norte (UFRN)（北里奥格兰德联邦大学（UFRN））；Mobile Brain（移动大脑（公司））；Institute of Psychiatry (IPUB)（精神病学研究所（IPUB））；Federal University of Rio de Janeiro (UFRJ)（里约热内卢联邦大学（UFRJ））；The University of Exeter（埃克塞特大学）；Institute for Data Science and Artificial Intelligence at the University of Exeter（埃克塞特大学数据科学与人工智能研究所）；The University of Sheffield（谢菲尔德大学）；Pontifical Catholic University of Rio Grande do Sul (PUCRS)（南里奥格兰德天主教大学（PUCRS））
Cardiff University（卡迪夫大学）；Iran University of Science and Technology（伊朗科技大学）
Over the past years, distributed semantic representations have proved to be effective and flexible keepers of prior knowledge to be integrated into downstream applications. This survey focuses on the representation of meaning. We start from the theoretical background behind word vector space models and highlight one of their major limitations: the meaning conflation deficiency, which arises from representing a word with all its possible meanings as a single vector. Then, we explain how this deficiency can be addressed through a transition from the word level to the more fine-grained level of w…
UC Berkeley（加州大学伯克利分校）；University of Vermont（佛蒙特大学）；The MITRE Corporation（MITRE公司）
Sentiment-aware intelligent systems are essential to a wide array of applications. These systems are driven by language models which broadly fall into two paradigms: Lexicon-based and contextual. Although recent contextual models are increasingly dominant, we still see demand for lexicon-based models because of their interpretability and ease of use. For example, lexicon-based models allow researchers to readily determine which words and phrases contribute most to a change in measured sentiment. A challenge for any lexicon-based approach is that the lexicon needs to be routinely expanded with …
Linguistic Features for Interpretable Textual Entailment
Centro de Investigación en Ciencias, Universidad Autónoma del Estado de Morelos（莫雷洛斯州自治大学科学研究中心）；Centro Interdisciplinario de Investigación en Humanidades, Universidad Autónoma del Estado de Morelos（莫雷洛斯州自治大学跨学科人文研究中心）
