精心设计好提示还是提供示范对话？基于角色对话生成的上下文学习研究
Crafting a Good Prompt or Providing Exemplary Dialogues? A Study of In-Context Learning for Persona-based Dialogue Generation
AI总结：
本研究在中文数据集上系统探究了大型语言模型在基于角色对话生成中的上下文学习能力，发现调整提示指令最有效，随机检索示例效果最佳，且模型能从损坏的示例中学习。
摘要：
先前的上下文学习（ICL）研究集中在分类、机器翻译、text2table等任务上，而关于ICL能否改进类人对话生成的研究却很匮乏。我们的工作填补了这一空白，在高质量的真实人类中文对话数据集上进行了广泛实验，系统研究了大型语言模型（LLMs）在基于角色对话生成中的ICL能力。从实验结果中，我们得出三个结论：1）调整提示指令是提高生成质量最直接、有效且经济的方法；2）随机检索示例（demos）取得了最佳结果，这可能归因于更大的多样性和有效信息量；与直觉相反，检索与查询上下文相同的示例表现最差；3）即使我们破坏示例中的多轮关联和单轮语义，增加示例数量仍能提升对话性能，证明LLMs能从损坏的对话示例中学习。以往对ICL机制的解释，如$n$-gram induction head，无法完全解释这一现象。
英文摘要：
Previous in-context learning (ICL) research has focused on tasks such as classification, machine translation, text2table, etc., while studies on whether ICL can improve human-like dialogue generation are scarce. Our work fills this gap by systematically investigating the ICL capabilities of large language models (LLMs) in persona-based dialogue generation, conducting extensive experiments on high-quality real human Chinese dialogue datasets. From experimental results, we draw three conclusions: 1) adjusting prompt instructions is the most direct, effective, and economical way to improve generation quality; 2) randomly retrieving demonstrations (demos) achieves the best results, possibly due to the greater diversity and the amount of effective information; counter-intuitively, retrieving demos with a context identical to the query performs the worst; 3) even when we destroy the multi-turn associations and single-turn semantics in the demos, increasing the number of demos still improves dialogue performance, proving that LLMs can learn from corrupted dialogue demos. Previous explanations of the ICL mechanism, such as $n$-gram induction head, cannot fully account for this phenomenon.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
