剖析复述：提示句法与补充信息对预训练语言模型知识检索的影响
Dissecting Paraphrases: The Impact of Prompt Syntax and supplementary Information on Knowledge Retrieval from Pretrained Language Models
AI总结：
本文提出包含3400万提示的探针 CONPARE-LAMA，系统比较复述句法与语义信息对预训练语言模型知识检索的独立影响，发现从句句法在多项指标上优于同位语句法，且范围信息比领域信息更能提升检索性能。
摘要：
预训练语言模型（PLMs）已知包含各种类型的知识。推断关系知识的一种方法是使用完形填空式提示，让模型预测缺失的主语或宾语。通常，设计这些提示是一项繁琐的任务，因为句法或语义上的细微差异都可能对知识检索性能产生重大影响。同时，由于提示句法与信息之间的相互依赖，评估其中任何一者的影响都具有挑战性。我们设计了 CONPARE-LAMA——一个专用探针，包含3400万个不同的提示，便于在最小复述之间进行比较。这些复述遵循统一的元模板，能够在任意关系上对句法和语义进行受控变化。CONPARE-LAMA 使得我们能够深入了解复述的句法形式或语义信息对 PLMs 知识检索性能的独立影响。使用我们的探针进行的大量知识检索实验表明，与同位语句法相比，采用从句句法的提示具有若干理想特性：i）在结合补充信息查询 PLMs 时更有用；ii）在不同补充信息组合下知识被更一致地回忆；iii）在检索已知事实时降低响应不确定性。此外，范围信息比领域信息更能提升知识检索性能，尽管领域信息在不同句法形式下更可靠地有帮助。
英文摘要：
Pre-trained Language Models (PLMs) are known to contain various kinds of knowledge. One method to infer relational knowledge is through the use of cloze-style prompts, where a model is tasked to predict missing subjects or objects. Typically, designing these prompts is a tedious task because small differences in syntax or semantics can have a substantial impact on knowledge retrieval performance. Simultaneously, evaluating the impact of either prompt syntax or information is challenging due to their interdependence. We designed CONPARE-LAMA - a dedicated probe, consisting of 34 million distinct prompts that facilitate comparison across minimal paraphrases. These paraphrases follow a unified meta-template enabling the controlled variation of syntax and semantics across arbitrary relations. CONPARE-LAMA enables insights into the independent impact of either syntactical form or semantic information of paraphrases on the knowledge retrieval performance of PLMs. Extensive knowledge retrieval experiments using our probe reveal that prompts following clausal syntax have several desirable properties in comparison to appositive syntax: i) they are more useful when querying PLMs with a combination of supplementary information, ii) knowledge is more consistently recalled across different combinations of supplementary information, and iii) they decrease response uncertainty when retrieving known facts. In addition, range information can boost knowledge retrieval performance more than domain information, even though domain information is more reliably helpful across syntactic forms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
