IERL：可解释的集成表示学习——结合众包知识与分布式语义表示
IERL: Interpretable Ensemble Representation Learning -- Combining CrowdSourced Knowledge and Distributed Semantic Representations
AI总结：
提出可解释的集成表示学习方法IERL，结合LLM与众包知识图谱，在GLUE任务上取得改进或具竞争力的结果，同时显著提升模型可解释性。
摘要：
大型语言模型（LLMs）以分布式语义的形式编码单词的含义。分布式语义从大量数据中捕获语言标记（单词、短语和句子）之间的常见统计模式。LLMs在旨在测试模型对输入标记含义理解能力的通用语言理解评估（GLUE）任务中表现极其优异。然而，最近的研究表明，当处理在训练期间很少见到的输入，或与多种上下文相关的输入（例如，语言生成任务中著名的幻觉现象）时，LLMs倾向于生成意外的、不一致的或错误的文本作为输出。诸如ConceptNet等众包和专家策划的知识图谱旨在从一组紧凑的、定义明确的上下文中捕获单词的含义。因此，LLMs可能会受益于利用此类知识上下文来减少输出中的不一致性。我们提出了一种新颖的集成学习方法，即可解释的集成表示学习（IERL），它系统地结合了输入标记的LLM和众包知识表示。与最先进（SOTA）的方法相比，IERL具有设计上可解释的独特优势（即何时使用了LLM上下文，何时使用了知识上下文？），允许结合模型参数对输入进行审查，从而促进对模型不一致或不相关输出的分析。尽管IERL对LLM和众包知识的选择不敏感，但我们使用BERT和ConceptNet演示了我们的方法。我们报告了IERL在GLUE任务中相比当前SOTA方法取得了改进或具有竞争力的结果，并显著增强了模型的可解释性。
英文摘要：
Large Language Models (LLMs) encode meanings of words in the form of distributed semantics. Distributed semantics capture common statistical patterns among language tokens (words, phrases, and sentences) from large amounts of data. LLMs perform exceedingly well across General Language Understanding Evaluation (GLUE) tasks designed to test a model's understanding of the meanings of the input tokens. However, recent studies have shown that LLMs tend to generate unintended, inconsistent, or wrong texts as outputs when processing inputs that were seen rarely during training, or inputs that are associated with diverse contexts (e.g., well-known hallucination phenomenon in language generation tasks). Crowdsourced and expert-curated knowledge graphs such as ConceptNet are designed to capture the meaning of words from a compact set of well-defined contexts. Thus LLMs may benefit from leveraging such knowledge contexts to reduce inconsistencies in outputs. We propose a novel ensemble learning method, Interpretable Ensemble Representation Learning (IERL), that systematically combines LLM and crowdsourced knowledge representations of input tokens. IERL has the distinct advantage of being interpretable by design (when was the LLM context used vs. when was the knowledge context used?) over state-of-the-art (SOTA) methods, allowing scrutiny of the inputs in conjunction with the parameters of the model, facilitating the analysis of models' inconsistent or irrelevant outputs. Although IERL is agnostic to the choice of LLM and crowdsourced knowledge, we demonstrate our approach using BERT and ConceptNet. We report improved or competitive results with IERL across GLUE tasks over current SOTA methods and significantly enhanced model interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
