MetaScale：基于演化元思维的测试时扩展
MetaScale: Test-Time Scaling with Evolving Meta-Thoughts
AI总结：
本文提出 METASCALE 测试时扩展框架，通过多臂老虎机选择与遗传算法演化任务自适应元思维，提升 LLM 推理准确率和泛化性。
摘要：
大型语言模型（LLM）进行复杂推理的一个关键挑战在于，它们依赖匹配训练数据中的推理模式，而不是主动选择最合适的认知策略来解决给定任务。现有方法施加固定的认知结构，虽然能提升特定任务的性能，却缺乏跨多样场景的适应性。为解决这一局限，我们提出 METASCALE，这是一个基于元思维（即为每项任务定制的适应性思维策略）的测试时扩展框架。METASCALE 先初始化一个候选元思维池，随后在奖励模型的引导下，利用采用上置信界选择的多臂老虎机算法迭代地选择并评估这些元思维。为进一步增强适应性，遗传算法会演化高奖励元思维，随时间不断精炼并扩展策略池。通过在推理时动态提出并优化元思维，METASCALE 在广泛任务上同时提升了准确性和泛化能力。实验结果表明，MetaScale 始终优于标准推理方法：在 Arena-Hard 上为 GPT-4o 带来 11% 的胜率性能提升，并在风格控制条件下以 0.9% 的优势超过 o1-mini。值得注意的是，随着采样预算增加，METASCALE 的扩展更为有效，并能生成更结构化、专家级的回答。
英文摘要：
One critical challenge for large language models (LLMs) for making complex reasoning is their reliance on matching reasoning patterns from training data, instead of proactively selecting the most appropriate cognitive strategy to solve a given task. Existing approaches impose fixed cognitive structures that enhance performance in specific tasks but lack adaptability across diverse scenarios. To address this limitation, we introduce METASCALE, a test-time scaling framework based on meta-thoughts -- adaptive thinking strategies tailored to each task. METASCALE initializes a pool of candidate meta-thoughts, then iteratively selects and evaluates them using a multi-armed bandit algorithm with upper confidence bound selection, guided by a reward model. To further enhance adaptability, a genetic algorithm evolves high-reward meta-thoughts, refining and extending the strategy pool over time. By dynamically proposing and optimizing meta-thoughts at inference time, METASCALE improves both accuracy and generalization across a wide range of tasks. Experimental results demonstrate that MetaScale consistently outperforms standard inference approaches, achieving an 11% performance gain in win rate on Arena-Hard for GPT-4o, surpassing o1-mini by 0.9% under style control. Notably, METASCALE scales more effectively with increasing sampling budgets and produces more structured, expert-level responses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
