大型语言模型生成中高级解码方法的语义不确定性
Semantic uncertainty in advanced decoding methods for LLM generation
AI总结：
本研究探究推测采样与思维链解码等高级解码方法对大型语言模型输出语义不确定性的影响，通过问答、摘要和代码生成实验发现，结构化解码可在保持或提升质量的同时增加语义多样性，如代码生成Pass@2提升48.8%，摘要生成ROUGE分数更优，挑战了多样性-准确性权衡的传统假设。
摘要：
本研究探讨了不同解码方法下大型语言模型（LLM）输出的语义不确定性，重点关注推测采样和思维链（CoT）解码等新兴技术。通过在问答、摘要生成和代码生成任务上的实验，我们分析了不同解码策略如何影响模型输出的多样性和可靠性。我们的研究结果显示，虽然思维链解码表现出更高的语义多样性，但其预测熵较低，这表明结构化探索可以带来更自信和准确的输出。这体现在代码生成Pass@2率提高了48.8%，尽管与参考解决方案的一致性较低。对于摘要生成任务，推测采样被证明特别有效，在保持中等语义多样性的同时取得了优越的ROUGE分数。我们的结果挑战了关于语言模型输出中多样性与准确性之间权衡的传统假设，证明适当结构的解码方法可以在保持或提高输出质量的同时增加语义探索。这些发现对于在实际应用中部署语言模型具有重要意义，在这些应用中，可靠性和多样化解决方案的生成都至关重要。
英文摘要：
This study investigates semantic uncertainty in large language model (LLM) outputs across different decoding methods, focusing on emerging techniques like speculative sampling and chain-of-thought (CoT) decoding. Through experiments on question answering, summarization, and code generation tasks, we analyze how different decoding strategies affect both the diversity and reliability of model outputs. Our findings reveal that while CoT decoding demonstrates higher semantic diversity, it maintains lower predictive entropy, suggesting that structured exploration can lead to more confident and accurate outputs. This is evidenced by a 48.8% improvement in code generation Pass@2 rates, despite lower alignment with reference solutions. For summarization tasks, speculative sampling proved particularly effective, achieving superior ROUGE scores while maintaining moderate semantic diversity. Our results challenge conventional assumptions about trade-offs between diversity and accuracy in language model outputs, demonstrating that properly structured decoding methods can increase semantic exploration while maintaining or improving output quality. These findings have significant implications for deploying language models in practical applications where both reliability and diverse solution generation are crucial.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
