iBERT:通过语义分解实现可解释嵌入
iBERT: Interpretable Embeddings via Sense Decomposition
AI总结:
iBERT通过语义分解生成可解释嵌入,提升风格表示效果并保持作者身份验证性能。
摘要:
我们提出了iBERT(可解释BERT),一种生成固有可解释和可控嵌入的编码器——旨在模块化并暴露语言中存在的判别线索,如语义或风格结构。每个输入标记被表示为k个上下文无关语义向量的稀疏非负混合,可以被池化为句子嵌入或直接在标记层面使用。这使在任何解码或下游使用之前都能对表示进行模块化控制。为了展示我们模型的可解释性,我们将其评估在一系列以风格为重点的任务上。在STEL基准测试中,它在风格表示有效性方面比SBERT风格基线提高了约8个点,同时在作者身份验证上保持具有竞争力的性能。因为每个嵌入都是可解释语义的结构组合,我们强调如何将特定的风格属性分配给特定的语义向量。虽然我们的实验集中在风格上,iBERT不限于风格建模。其结构模块化设计旨在可解释地分解数据中存在的任何判别信号——即使监督融合了语义或风格因素,也能实现泛化。
英文摘要:
We present iBERT (interpretable-BERT), an encoder to produce inherently interpretable and controllable embeddings - designed to modularize and expose the discriminative cues present in language, such as semantic or stylistic structure. Each input token is represented as a sparse, non-negative mixture over k context-independent sense vectors, which can be pooled into sentence embeddings or used directly at the token level. This enables modular control over representation, before any decoding or downstream use. To demonstrate our model's interpretability, we evaluate it on a suite of style-focused tasks. On the STEL benchmark, it improves style representation effectiveness by ~8 points over SBERT-style baselines, while maintaining competitive performance on authorship verification. Because each embedding is a structured composition of interpretable senses, we highlight how specific style attributes get assigned to specific sense vectors. While our experiments center on style, iBERT is not limited to stylistic modeling. Its structural modularity is designed to interpretably decompose whichever discriminative signals are present in the data - enabling generalization even when supervision blends semantic or stylistic factors.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。