High-dimensional representations for words, text, images, knowledge graphs and other structured data are commonly used in different paradigms of machine learning and data mining. These representations have different degrees of interpretability, with efficient distributed representations coming at the cost of the loss of feature to dimension mapping. This implies that there is obfuscation in the way concepts are captured in these embedding spaces. Its effects are seen in many representations and tasks, one particularly problematic one being in language representations where the societal biases, learned from underlying data, are captured and occluded in unknown dimensions and subspaces. As a result, invalid associations (such as different races and their association with a polar notion of good versus bad) are made and propagated by the representations, leading to unfair outcomes in different tasks where they are used. This work addresses some of these problems pertaining to the transparency and interpretability of such representations. A primary focus is the detection, quantification, and mitigation of socially biased associations in language representation.
Atlas-Alignment: Making Interpretability Transferable Across Language Models
Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute（人工智能系，弗劳恩霍夫 Heinrich Hertz 研究所）；Department of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学）；Centre of eXplainable Artificial Intelligence, Technological University Dublin（可解释人工智能中心，都柏林技术大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（BIFOLD - 柏林学习与数据基础研究所）
SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Institute for AI, Peking University, Beijing, China（人工智能研究院，北京大学，北京，中国）；State Key Laboratory of General Artificial Intelligence, Institute for AI, Peking University, Beijing, China（通用人工智能国家重点实验室，人工智能研究院，北京大学，北京，中国）
East China Normal University（华东师范大学）；Shanghai Institute of AI for Education（上海智能教育研究院）；TikTok Group（TikTok集团）
Entity Alignment (EA) aims to find equivalent entity pairs between KGs, which is the core step of bridging and integrating multi-source KGs. In this paper, we argue that existing GNN-based EA methods inherit the inborn defects from their neural network lineage: weak scalability and poor interpretability. Inspired by recent studies, we reinvent the Label Propagation algorithm to effectively run on KGs and propose a non-neural EA framework -- LightEA, consisting of three efficient components: (i) Random Orthogonal Label Generation, (ii) Three-view Label Propagation, and (iii) Sparse Sinkhorn Ite…
FADE: Why Bad Descriptions Happen to Good Features
Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute（人工智能系，弗劳恩霍夫海因里希·赫兹研究所）；Department of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Centre of eXplainable Artificial Intelligence, Technological University Dublin（可解释人工智能中心，都柏林技术大学）
Unlocking LLM Safeguards for Low-Resource Languages via Reasoning and Alignment with Minimal Training Data
Guangdong University of Foreign Studies（广东外语外贸大学）；Guangzhou Key Laboratory of Multilingual Intelligent Processing（广州多语智能处理重点实验室）；University of Pittsburgh（匹兹堡大学）
A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models
Department of Computer Science and Technology（计算机科学与技术系）；Cancer Research UK（癌症研究英国公司）；Cambridge Institute（剑桥研究所）；University of Cambridge（剑桥大学）；United Kingdom（英国）；DIMES（迪梅斯）；University of Calabria（卡拉布里亚大学）；Italy（意大利）；Department of Computer Automatic and Management Engineering (DIAG)（计算机自动与管理工程系）；Sapienza Università di Roma（罗马大学萨皮恩扎）；Department of Psychiatry（精神病学系）；School of Computing（计算学院）；University of Kent（肯特大学）；Department of Psychology（心理学系）
Previous works have demonstrated the effectiveness of utilising pre-trained sentence encoders based on their sentence representations for meaning comparison tasks. Though such representations are shown to capture hidden syntax structures, the direct similarity comparison between them exhibits weak sensitivity to word order and structural differences in given sentences. A single similarity score further makes the comparison process hard to interpret. Therefore, we here propose to combine sentence encoders with an alignment component by representing each sentence as a list of predicate-argument …
ASELSAN Research Center（阿瑟兰研究中心）；Bilkent University（比尔肯特大学）；Ludwig Maximillian University（路德维希·马克西米利安大学）
As an ubiquitous method in natural language processing, word embeddings are extensively employed to map semantic properties of words into a dense vector representation. They capture semantic and syntactic relations among words but the vectors corresponding to the words are only meaningful relative to each other. Neither the vector nor its dimensions have any absolute, interpretable meaning. We introduce an additive modification to the objective function of the embedding learning algorithm that encourages the embedding vectors of words that are semantically related to a predefined concept to ta…
Graduate Institute of International and Development Studies（日内瓦高级国际关系及发展学院）；Technical University of Munich（慕尼黑工业大学）
This work demonstrates that Legal Judgement Prediction systems without expert-informed adjustments can be vulnerable to shallow, distracting surface signals that arise from corpus construction, case distribution, and confounding factors. To mitigate this, we use domain expertise to strategically identify statistically predictive but legally irrelevant information. We adopt adversarial training to prevent the system from relying on it. We evaluate our deconfounded models by employing interpretability techniques and comparing to expert annotations. Quantitative experiments and qualitative analys…
Explaining Intrinsic Moral Self-Correction with Mechanistic Interpretability
Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan（通讯工程研究院，国立台湾大学）；MediaTek Research, Taipei, Taiwan（联发科研究，台北，台湾）；Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan（电气工程系，国立台湾大学）；Department of Electrical Engineering, National Tsing Hua University, Hsinchu, Taiwan（电气工程系，国立清华大学）；AI Research Center (AINTU), National Taiwan University, Taipei, Taiwan（人工智能研究中心（AINTU），国立台湾大学）
From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop
Meta；Autodesk（欧特克公司）；New Jersey Institute of Technology（新泽西理工学院）；Salesforce（salesforce公司）；University of California, Los Angeles（加州大学洛杉矶分校）；Amazon AGI（亚马逊AGI）
