论 AMR 增强对大语言模型（无效）效果
On the (In)effectiveness of AMR Augmentation for Large Language Models
- Saarland University（萨尔大学）
- University of Trento（特伦托大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文通过复现实验发现 AMR 增强对 LLM 下游任务无益，并引入困惑度探针证实其未提升关系理解，表明该增强无效。
AI中文摘要：
尽管抽象意义表示（AMR）历来在多种自然语言处理任务上提升了性能，但对于现代大语言模型（LLMs）而言，AMR 增强的益处（或缺乏益处）迄今尚不明确。本文中，我们尝试复现近期报道 AMR 增强带来显著下游收益的研究，发现这些收益很可能源于实验设置中的特定选择：采用一致且统一的超参数选择协议，我们观察到仅使用文本的基线模型始终匹配或超越 AMR 增强模型的性能。为探究这一零结果，我们引入一种基于困惑度的探针，用于衡量 AMR 为 LLM 提供的补充关系知识（模型本身无法获得）的程度。我们发现 AMR 增强并未帮助 LLM 提升对句子中关系内容的理解，表明对这些模型进行 AMR 增强在下游任务中并无明确益处。
英文摘要：
While Abstract Meaning Representation (AMR) has historically improved performance on a range of NLP tasks, the benefit---or lack thereof---of AMR augmentation for modern LLMs is thus far unclear. In this paper, we attempt to reproduce recent work that reported substantial downstream gains from AMR augmentation, finding that these are likely due to specific choices in the experimental settings used: using a consistent and unified protocol for hyperparameter selection, we observe that text-only baselines consistently match or exceed the performance of AMR-augmented models. To investigate this null result, we introduce a perplexity-based probe measuring the degree to which AMR provides an LLM with supplemental relational knowledge not already available to the model. We find that AMR augmentation does not help LLMs improve their understanding of relational content in the sentence, indicating that augmenting these models with AMR offers no clear benefit on downstream tasks.
