THiNK：大型语言模型能否进行有声思考？
THiNK: Can Large Language Models Think-aloud?
AI总结：
本文提出THiNK，一个基于布鲁姆分类法的多智能体反馈驱动评估框架，通过迭代生成、批评和修订促使LLM进行有声思考，系统评估低阶与高阶思维技能，实验表明反馈循环显著提升高阶推理性能。
摘要：
评估大型语言模型（LLMs）中的高阶思维技能仍然是一个根本性挑战，尤其是在超越表面准确性的任务中。在这项工作中，我们提出了THiNK（测试高阶知识概念），一个基于布鲁姆分类法的多智能体、反馈驱动的评估框架。THiNK将推理评估构建为问题生成、批评和修订的迭代任务，鼓励LLMs通过逐步反思和细化进行有声思考。这使得能够系统评估低阶（例如，记忆、理解）和高阶（例如，评价、创造）思维技能。我们将THiNK应用于七个最先进的LLMs，并对其输出进行了详细的认知分析。结果显示，虽然模型在低阶类别上表现可靠，但在实际情境中应用知识方面存在困难，并且表现出有限的抽象能力。结构化反馈循环显著提高了推理性能，特别是在高阶思维方面。定性评估进一步证实，THiNK引导的输出更好地符合领域逻辑和问题结构。我们框架的代码提供了一种可扩展的方法来探测和增强LLM推理，为基于学习科学的评估提供了新方向，该代码可在我们的GitHub仓库中获取。
英文摘要：
Assessing higher-order thinking skills in large language models (LLMs) remains a fundamental challenge, especially in tasks that go beyond surface-level accuracy. In this work, we propose THiNK (Testing Higher-order Notion of Knowledge), a multi-agent, feedback-driven evaluation framework grounded in Bloom's Taxonomy. THiNK frames reasoning assessment as an iterative task of problem generation, critique, and revision, encouraging LLMs to think-aloud through step-by-step reflection and refinement. This enables a systematic evaluation of both lower-order (e.g., remember, understand) and higher-order (e.g., evaluate, create) thinking skills. We apply THiNK to seven state-of-the-art LLMs and perform a detailed cognitive analysis of their outputs. Results reveal that while models reliably perform lower-order categories well, they struggle with applying knowledge in realistic contexts and exhibit limited abstraction. Structured feedback loops significantly improve reasoning performance, particularly in higher-order thinking. Qualitative evaluations further confirm that THiNK-guided outputs better align with domain logic and problem structure. The code of our framework provides a scalable methodology for probing and enhancing LLM reasoning, offering new directions for evaluation grounded in learning science, which is available at our GitHub repository.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
