探究相关知识对自然语言生成可解释性的影响
Exploring the Influence of Relevant Knowledge for Natural Language Generation Interpretability
AI总结：
该研究聚焦自然语言生成的常识生成任务，构建KITGI基准，通过对比完整与过滤高相关知识下T5-Large模型的生成效果，证实相关外部知识对维持生成连贯性和概念覆盖率至关重要，为可解释知识增强NLG系统设计提供依据。
摘要：
本文探究外部知识整合在自然语言生成（Natural Language Generation, NLG）中的影响，聚焦常识生成任务。我们对CommonGen数据集进行扩展，构建了KITGI基准，该基准将输入概念集与从ConceptNet检索到的语义关系配对，并包含人工标注的输出。我们使用T5-Large模型，对比两种条件下的句子生成效果：具备完整外部知识的条件，以及刻意移除高相关关系的过滤知识条件。我们的可解释性基准遵循三阶段方法：（1）识别并移除关键知识；（2）重新生成句子；（3）人工评估输出的常识合理性和概念覆盖率。结果显示，使用完整知识生成的句子在两项标准上的正确率均达到91%，而过滤知识后性能大幅下降至6%。这些发现表明，相关外部知识对于维持NLG的连贯性和概念覆盖率至关重要。本工作凸显了设计可解释的知识增强NLG系统的重要性，并呼吁建立能够捕捉表层指标之外底层推理的评估框架。
英文摘要：
This paper explores the influence of external knowledge integration in Natural Language Generation (NLG), focusing on a commonsense generation task. We extend the CommonGen dataset by creating KITGI, a benchmark that pairs input concept sets with retrieved semantic relations from ConceptNet and includes manually annotated outputs. Using the T5-Large model, we compare sentence generation under two conditions: with full external knowledge and with filtered knowledge where highly relevant relations were deliberately removed. Our interpretability benchmark follows a three-stage method: (1) identifying and removing key knowledge, (2) regenerating sentences, and (3) manually assessing outputs for commonsense plausibility and concept coverage. Results show that sentences generated with full knowledge achieved 91\% correctness across both criteria, while filtering reduced performance drastically to 6\%. These findings demonstrate that relevant external knowledge is critical for maintaining both coherence and concept coverage in NLG. This work highlights the importance of designing interpretable, knowledge-enhanced NLG systems and calls for evaluation frameworks that capture the underlying reasoning beyond surface-level metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
