arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.02246cs.CLcs.AI
收藏

DETAIL 重要性:测量提示特定性对大语言模型推理的影响

DETAIL Matters: Measuring the Impact of Prompt Specificity on Reasoning in Large Language Models

Emory University(埃默里大学)

Olivia Kim

AI总结:

本文通过DETAIL框架研究提示特定性对大语言模型推理性能的影响,发现特定性提升准确性,尤其对小型模型和程序性任务效果显著。

摘要:

提示设计在大语言模型(LLMs)的推理性能中起着关键作用,然而提示特定性——提示的详细程度或模糊程度——的影响仍缺乏研究。本文介绍了DETAIL框架,用于评估LLM在不同提示特定性水平下的表现。我们使用GPT-4生成多层次提示,通过困惑度量化特定性,并利用基于GPT的语义等价性评估正确性。在30个新型推理任务上对GPT-4和O3-mini进行实验,发现特定性提高准确性,尤其是对较小模型和程序性任务。我们的结果强调了需要适应性提示策略,并提供了工具和数据以支持进一步研究。

英文摘要:

Prompt design plays a critical role in the reasoning performance of large language models (LLMs), yet the impact of prompt specificity - how detailed or vague a prompt is - remains understudied. This paper introduces DETAIL, a framework for evaluating LLM performance across varying levels of prompt specificity. We generate multi-level prompts using GPT-4, quantify specificity via perplexity, and assess correctness using GPT-based semantic equivalence. Experiments on 30 novel reasoning tasks across GPT-4 and O3-mini reveal that specificity improves accuracy, especially for smaller models and procedural tasks. Our results highlight the need for adaptive prompting strategies and provide tools and data to support further research.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑