重新思考不可解问题：当上下文搜索遇上测试时扩展
Rethinking the Unsolvable: When In-Context Search Meets Test-Time Scaling
AI总结：
本文探讨了上下文搜索与测试时扩展结合在超难推理任务上的潜力，通过实证与理论分析证明该方法能显著提升成功率并扩展可解问题复杂度类别，挑战了现有LLM评估范式。
摘要：
近期研究强调，大型语言模型（LLMs）即使在经过训练以生成扩展的冗长推理步骤后，在处理困难推理问题时仍面临重大挑战。然而，现有文献大多依赖直接提示和简单的上下文学习示例进行评估，这在很大程度上忽略了采用先进技术在得出LLMs触及性能天花板的结论之前，来激发其深思熟虑的推理能力。在本文中，我们系统地探讨了上下文搜索与测试时扩展在超难推理任务上的结合潜力。我们发现，通过对经过内部扩展增强的LLMs采用先进的上下文搜索提示，能够在以前被认为“不可解”的任务上（例如，报告成功率低于5%）实现变革性的性能突破。我们提供了关于这种结合如何释放LLM推理能力的实证结果和理论分析：i）在实证方面，在受控的NP难任务和复杂的现实世界规划基准上，我们的方法与先前报告的结果相比，在没有任何外部机制的情况下实现了高达30倍的成功率提升；ii）在理论方面，我们证明了上下文搜索提示与内部扩展相结合时，显著扩展了可解推理问题的复杂度类别。这些发现挑战了关于LLMs在复杂任务上局限性的普遍假设，表明当前的评估范式系统性地低估了它们的真实潜力。我们的工作呼吁对LLM推理的基准测试进行批判性重新评估，并采用更稳健的评估策略以充分捕捉当代LLMs的真实能力，这有助于更好地理解它们在现实部署中的操作推理边界。
英文摘要：
Recent research has highlighted that Large Language Models (LLMs), even when trained to generate extended long reasoning steps, still face significant challenges on hard reasoning problems. However, much of the existing literature relies on direct prompting with simple in-context learning examples for evaluation, which largely overlooks advanced techniques to elicit LLMs' deliberate reasoning before drawing conclusions that LLMs hit a performance ceiling. In this paper, we systematically explore the combined potential of in-context search and test-time scaling on super hard reasoning tasks. We find that by employing advanced in-context search prompting to LLMs augmented with internal scaling, one can achieve transformative performance breakthroughs on tasks previously deemed "unsolvable" (e.g., reported success rates below 5%). We provide both empirical results and theoretical analysis of how this combination can unleash LLM reasoning capabilities: i) Empirically, on controlled NP-hard tasks and complex real-world planning benchmarks, our approach achieves up to a 30x improvement in success rates compared to previously reported results without any external mechanisms; ii) Theoretically, we show that in-context search prompting, when combined with internal scaling, significantly extends the complexity class of solvable reasoning problems. These findings challenge prevailing assumptions about the limitations of LLMs on complex tasks, indicating that current evaluation paradigms systematically underestimate their true potential. Our work calls for a critical reassessment of how LLM reasoning is benchmarked and a more robust evaluation strategy that fully captures the true capabilities of contemporary LLMs, which can lead to a better understanding of their operational reasoning boundaries in real-world deployments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
