在信息瓶颈视角下,大型语言模型自我解释的充分性与简洁性权衡
The Sufficiency-Conciseness Trade-off in LLM Self-Explanation from an Information Bottleneck Perspective
AI总结:
本文从信息瓶颈视角探讨了大型语言模型自我解释中充分性与简洁性之间的权衡,通过实验表明简洁解释在保持准确性的同时能显著减少长度,但过度压缩会损害性能。
摘要:
大型语言模型越来越多地依赖自我解释,如推理链,以提高在多步问答任务上的性能。尽管这些解释提高了准确性,但它们通常冗长且生成成本高,引发了关于真正需要多少解释的问题。在本文中,我们研究了充分性(解释能够证明正确答案的能力)与简洁性(解释长度的减少)之间的权衡。基于信息瓶颈原理,我们将解释视为压缩表示,只保留产生正确答案所需的信息。为了将这一观点具体化,我们引入了一个评估流程,限制解释长度,并使用多个语言模型在ARC挑战数据集上评估充分性。为了扩大范围,我们同时在英语和波斯语上进行实验,后者通过翻译作为资源受限语言。我们的实验表明,更简洁的解释往往仍能保持充分性,从而在大幅减少解释长度的同时保持准确性,而过度压缩会导致性能下降。
英文摘要:
Large Language Models increasingly rely on self-explanations, such as chain of thought reasoning, to improve performance on multi step question answering. While these explanations enhance accuracy, they are often verbose and costly to generate, raising the question of how much explanation is truly necessary. In this paper, we examine the trade-off between sufficiency, defined as the ability of an explanation to justify the correct answer, and conciseness, defined as the reduction in explanation length. Building on the information bottleneck principle, we conceptualize explanations as compressed representations that retain only the information essential for producing correct answers.To operationalize this view, we introduce an evaluation pipeline that constrains explanation length and assesses sufficiency using multiple language models on the ARC Challenge dataset. To broaden the scope, we conduct experiments in both English, using the original dataset, and Persian, as a resource-limited language through translation. Our experiments show that more concise explanations often remain sufficient, preserving accuracy while substantially reducing explanation length, whereas excessive compression leads to performance degradation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。