影响策略重要吗？研究大语言模型代码生成中的提示词框架效应
Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation
浏览论文内容
中文总结 AI 辅助
本研究首次大规模实证探究基于心理学的影响策略诱导的提示词框架对LLM代码生成的影响，发现强调紧迫性的框架会降低代码正确性与安全性，为设计人机交互提供实践见解。
中文摘要 AI 辅助
大语言模型（LLM）正越来越多地被集成到软件工程工作流中，帮助开发者编写、调试、测试和维护代码。虽然提示词的措辞和结构已知会影响模型性能，但受心理学启发的提示词框架的影响仍未被探索。本研究调查人类用于说服或激励他人的不同基于心理学的沟通策略是否能带来更有效的提示词框架，这可能反过来影响LLM在编码任务中的行为。借鉴Yukl & Falbe的著名分类法，我们将8种影响策略（如理性说服、讨好和交换）操作化为可复现的提示词模板。这些提示词模板在5种领先的开放权重LLM上使用两个广泛采用的基准：LiveCodeBench和SWE-bench Verified进行评估。我们在四个关键软件质量维度上评估生成的代码输出：功能正确性、质量、可维护性和安全性。我们的结果表明，某些由影响策略诱导的提示词框架，特别是那些强调紧迫性的框架，与正确性和安全性降低相关。本研究是软件工程任务中由影响策略诱导的提示词框架的首次大规模实证研究，提供了语言线索如何塑造LLM输出的见解。我们最后提出了设计代码生成中透明且可解释的人机交互的实践见解。
英文摘要
Large Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code. While prompt wording and structure are known to influence model performance, the impact of psychologically inspired prompt framings remains unexplored. This study investigates whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective prompt framing, which may, in turn, affect LLM behaviour in coding tasks. Drawing on Yukl & Falbe's well-known taxonomy, we operationalized eight influence tactics (like rational persuasion, ingratiation, and exchange) into reproducible prompt templates. These prompt templates were evaluated across five leading open-weight LLMs using two widely adopted benchmarks: LiveCodeBench and SWE-bench Verified. We assessed the resulting code output on four key software quality dimensions: functional correctness, quality, maintainability, and security. Our results show that certain influence-induced prompt framings, particularly those emphasizing urgency, were associated with reduced correctness and security. This work presents the first large-scale empirical study of influence-induced prompt framing in software engineering tasks, offering insights into how linguistic cues may shape LLM outputs. We conclude with practical insights for designing transparent and interpretable human-AI interactions in code generation.
发表机构
- Extended author information available on the last page of the article
机构由 AI 辅助整理，请以论文原文为准。
