Explainable Semantic Textual Similarity via Dissimilar Span Detection
School of Computation, Information and Technology（计算、信息与技术学院）；Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Diego Miguel Lozano, Daryna Dementieva, Alexander Fraser
Semantic Textual Similarity (STS) is a crucial component of many Natural Language Processing (NLP) applications. However, existing approaches typically reduce semantic nuances to a single score, limiting interpretability. To address this, we introduce the task of Dissimilar Span Detection (DSD), which aims to identify semantically differing spans between pairs of texts. This can help users understand which particular words or tokens negatively affect the similarity score, or be used to improve performance in STS-dependent downstream tasks. Furthermore, we release a new dataset suitable for the task, the Span Similarity Dataset (SSD), developed through a semi-automated pipeline combining large language models (LLMs) with human verification. We propose and evaluate different baseline methods for DSD, both unsupervised, based on LIME, SHAP, LLMs, and our own method, as well as an additional supervised approach. While LLMs and supervised models achieve the highest performance, overall results remain low, highlighting the complexity of the task. Finally, we set up an additional experiment that shows how DSD can lead to increased performance in the specific task of paraphrase detection.
Despite the success achieved on various natural language processing tasks, word embeddings are difficult to interpret due to the dense vector representations. This paper focuses on interpreting the embeddings for various aspects, including sense separation in the vector dimensions and definition generation. Specifically, given a context together with a target word, our algorithm first projects the target word embedding to a high-dimensional sparse vector and picks the specific dimensions that can best explain the semantic meaning of the target word by the encoded contextual information, where …
The University of Sheffield（谢菲尔德大学）；Technische Universität Darmstadt（达姆施塔特工业大学）；The University of Bath（巴斯大学）
We compare sequential fine-tuning with a model for multi-task learning in the context where we are interested in boosting performance on two tasks, one of which depends on the other. We test these models on the FigLang2022 shared task which requires participants to predict language inference labels on figurative language along with corresponding textual explanations of the inference predictions. Our results show that while sequential multi-task learning can be tuned to be good at the first of two target tasks, it performs less well on the second and additionally struggles with overfitting. Our…
University of Alberta（阿尔伯塔大学）；Carnegie Mellon University（卡内基梅隆大学）
Natural language inference (NLI) aims to determine the logical relationship between two sentences, such as Entailment, Contradiction, and Neutral. In recent years, deep learning models have become a prevailing approach to NLI, but they lack interpretability and explainability. In this work, we address the explainability of NLI by weakly supervised logical reasoning, and propose an Explainable Phrasal Reasoning (EPR) approach. Our model first detects phrases as the semantic unit and aligns corresponding phrases in the two sentences. Then, the model predicts the NLI label for the aligned phrases…
Despite the high accuracy offered by state-of-the-art deep natural-language models (e.g. LSTM, BERT), their application in real-life settings is still widely limited, as they behave like a black-box to the end-user. Hence, explainability is rapidly becoming a fundamental requirement of future-generation data-driven systems based on deep-learning approaches. Several attempts to fulfill the existing gap between accuracy and interpretability have been done. However, robust and specialized xAI (Explainable Artificial Intelligence) solutions tailored to deep natural-language models are still missin…
An increasing number of people are using online social networking services (SNSs), and a significant amount of information related to experiences in consumption is shared in this new media form. Text mining is an emerging technique for mining useful information from the web. We aim at discovering in particular tweets semantic patterns in consumers' discussions on social media. Specifically, the purposes of this study are twofold: 1) finding similarity and dissimilarity between two sets of textual documents that include consumers' sentiment polarities, two forms of positive vs. negative opinion…
MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrieval
University of Sheffield（谢菲尔德大学）；University of Science and Technology Beijing（北京科技大学）；University of California San Diego（加州大学圣地亚哥分校）；The Pennsylvania State University（宾夕法尼亚州立大学）
Explainable deep learning models are advantageous in many situations. Prior work mostly provide unimodal explanations through post-hoc approaches not part of the original system design. Explanation mechanisms also ignore useful textual information present in images. In this paper, we propose MTXNet, an end-to-end trainable multimodal architecture to generate multimodal explanations, which focuses on the text in the image. We curate a novel dataset TextVQA-X, containing ground truth visual and multi-reference textual explanations that can be leveraged during both training and evaluation. We the…
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
The ability of reasoning over evidence has received increasing attention in question answering (QA). Recently, natural language database (NLDB) conducts complex QA in knowledge base with textual evidences rather than structured representations, this task attracts a lot of attention because of the flexibility and richness of textual evidence. However, existing text-based complex question answering datasets fail to provide explicit reasoning process, while it's important for retrieval effectiveness and reasoning interpretability. Therefore, we present a benchmark \textbf{ReasonChainQA} with expl…
Explainable Multimodal Aspect-Based Sentiment Analysis with Dependency-guided Large Language Model
Harbin Institute of Technology（哈尔滨工程大学）；Zhongguancun Academy（中关村学院）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）；University of Science and Technology of China（中国科学技术大学）
Georgia Institute of Technology（佐治亚理工学院）；Allen Institute for AI（艾伦人工智能研究所）；University of Washington（华盛顿大学）
Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as supervision to train models to produce explanations for their predictions, and as a ground-truth to evaluate model-generated explanations. In this review, we identify 65 datasets with three predominant classes of textual explanations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing coll…
In this article, we first give an introduction to artificial intelligence and its applications in biology and medicine in Section 1. Deep learning methods are then described in Section 2. We narrow down the focus of the study on textual data in Section 3, where natural language processing and its applications in the biomedical domain are described. In Section 4, we give an introduction to explainable artificial intelligence and discuss the importance of explainability of artificial intelligence systems, especially in the biomedical domain.
UC Berkeley（加州大学伯克利分校）；University of Amsterdam（阿姆斯特丹大学）；MPI for Informatics（马克斯·普朗克信息学研究所）；Facebook AI Research（脸书人工智能研究院）
Deep models that are both effective and explainable are desirable in many settings; prior explainable models have been unimodal, offering either image-based visualization of attention weights or text-based generation of post-hoc justifications. We propose a multimodal approach to explanation, and argue that the two modalities provide complementary explanatory strengths. We collect two new datasets to define and evaluate this task, and propose a novel model which can provide joint textual rationale generation and attention visualization. Our datasets define visual and textual justifications of …
SAMOVAR；Télécom SudParis（巴黎南电信学院）；Institut Polytechnique de Paris（巴黎综合理工学院）
When a new computer security vulnerability is publicly disclosed, only a textual description of it is available. Cybersecurity experts later provide an analysis of the severity of the vulnerability using the Common Vulnerability Scoring System (CVSS). Specifically, the different characteristics of the vulnerability are summarized into a vector (consisting of a set of metrics), from which a severity score is computed. However, because of the high number of vulnerabilities disclosed everyday this process requires lot of manpower, and several days may pass before a vulnerability is analyzed. We p…
Georgia Institute of Technology（佐治亚理工学院）；Allen Institute for AI（艾伦人工智能研究所）
We focus on the task of creating a reinforcement learning agent that is inherently explainable -- with the ability to produce immediate local explanations by thinking out loud while performing a task and analyzing entire trajectories post-hoc to produce causal explanations. This Hierarchically Explainable Reinforcement Learning agent (HEX-RL), operates in Interactive Fictions, text-based game environments in which an agent perceives and acts upon the world using textual natural language. These games are usually structured as puzzles or quests with long-term dependencies in which an agent must …
Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study
Technical University of Munich, School of Computation, Information and Technology, Department of Computer Science, Munich, Germany（慕尼黑技术大学，计算、信息与技术学院，计算机科学系，德国慕尼黑）
UkrMeme: Multimodal Explainable Persuasion Detection in Ukrainian Memes
Ukrainian Catholic University（乌克兰天主教大学）；Johannes Gutenberg University Mainz（美因茨约翰内斯·古滕贝格大学）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Centre for European Research in Trusted AI (CERTAIN)（欧洲可信人工智能研究中心）
University of Waterloo（滑铁卢大学）；The Central Bank of the Republic of Türkiye（土耳其共和国中央银行）；Bilkent University（比尔肯特大学）
Translating natural language queries (NLQ) into structured query language (SQL) in interfaces to relational databases is a challenging task that has been widely studied by researchers from both the database and natural language processing communities. Numerous works have been proposed to attack the natural language interfaces to databases (NLIDB) problem either as a conventional pipeline-based or an end-to-end deep-learning-based solution. Nevertheless, regardless of the approach preferred, such solutions exhibit black-box nature, which makes it difficult for potential users targeted by these …
Fraunhofer FIT（弗劳恩霍夫应用信息技术研究所）；RWTH Aachen University（亚琛工业大学）；Noakhali Science and Technology University（诺亚克利科技大学）；Begum Rokeya University（贝古姆·罗凯亚大学）；The University of Alabama（阿拉巴马大学）；National University of Ireland, Galway（爱尔兰国立高威大学）；Islamic University of Technology（伊斯兰科技大学）
The exponential growths of social media and micro-blogging sites not only provide platforms for empowering freedom of expressions and individual voices, but also enables people to express anti-social behaviour like online harassment, cyberbullying, and hate speech. Numerous works have been proposed to utilize textual data for social and anti-social behaviour analysis, by predicting the contexts mostly for highly-resourced languages like English. However, some languages are under-resourced, e.g., South Asian languages like Bengali, that lack computational resources for accurate natural language…
Uncertainty-Aware Explainable Recommendation with Large Language Models
Chengdu University of Information Technology（成都信息工程大学）；Tunghai University（东海大学）；Leshan Normal University（乐山师范学院）；University at Buffalo, SUNY（纽约州立大学布法罗分校）；Meta；Purdue University（普渡大学）；University at Albany, SUNY（纽约州立大学奥尔巴尼分校）
