arXiv 2606.116782026-07-14cs.CL

AI能像城市规划师一样推理吗？基于专业判断的大语言模型基准测试

Can AI Reason Like an Urban Planner? Benchmarking Large Language Models Against Professional Judgment

School of Architecture and Urban Planning, Shenzhen University（深圳大学建筑与城市规划学院）；Shenzhen Key Laboratory of Urban Spatial Information and Intelligent Modeling（深圳市城市空间信息与智能建模重点实验室）；Department of Urban Planning and Design, The University of Hong Kong（香港大学城市规划与设计系）

Yijie Deng, He Zhu, Wen Wang, Junyou Su, Minxin Chen, Wenjia Zhang

备注：: This paper has been withdrawn by the authors because the current version requires substantial revision and further validation before it can be considered a reliable representation of the work

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出UPBench框架，通过4×5知识支柱与认知水平矩阵评估25个LLM，发现模型在分析任务上优于事实回忆和综合判断，揭示了规划知识的制度依赖性。

摘要：

问题、研究策略与发现：大语言模型（LLM）的兴起为城市规划提出了一个关键问题：AI能复制哪些专业规划知识，哪些仍需人类判断？尽管AI工具在规划实践中日益普及，但目前仍缺乏系统性框架来测试它们是否能以规划专业知识核心的情境敏感性、价值意识和制度素养进行推理。本文介绍了Urban Planning Bench（UPBench），这是一个领域特定的评估框架，通过改编自布鲁姆修订分类法的四个知识支柱和五个认知水平构成的4x5矩阵来评估LLM推理。通过自动评分和专家评审对25个LLM进行评估，我们发现了一条非单调的认知曲线：模型在高级分析任务上的表现优于事实回忆和综合判断。这表明，通常被视为低阶的规划知识深受制度、司法和时间背景的影响，使得LLM难以泛化。我们将这些局限性总结为四个认知诊断：监管幻觉、概念混淆、棘手性瘫痪和实践智慧缺陷。实践启示：研究结果支持规划中的差异化委托。LLM可以协助跨学科综合、文献综述、情景生成和初步政策分析。然而，它们在特定司法管辖区的法规、规范冲突解决和情境敏感程序方面仍不可靠。机构应要求对AI辅助监管分析进行验证，而规划教育应强调制度素养、规范判断和情境敏感性。

英文摘要：

Problem, Research Strategy, and Findings: The rise of large language models (LLMs) raises a key question for urban planning: which forms of professional planning knowledge can AI replicate, and which still require human judgment? Although AI tools are increasingly used in planning practice, there is still no systematic framework for testing whether they can reason with the contextual sensitivity, value awareness, and institutional literacy central to planning expertise. This paper introduces Urban Planning Bench (UPBench), a domain-specific evaluation framework that assesses LLM reasoning through a 4x5 matrix of four knowledge pillars and five cognitive levels adapted from Bloom's revised taxonomy. Evaluating 25 LLMs with automated scoring and expert review, we find a non-monotonic cognitive curve: models perform better on higher-order analytical tasks than on factual recall and integrative judgment. This suggests that planning knowledge often treated as lower-order is deeply shaped by institutional, jurisdictional, and temporal context, making it hard for LLMs to generalize. We summarize these limits as four epistemic diagnostics: regulatory hallucination, conceptual conflation, wickedness paralysis, and phronetic deficit. Takeaway for Practice: The findings support differential delegation in planning. LLMs can assist with cross-disciplinary synthesis, literature review, scenario generation, and preliminary policy analysis. However, they remain unreliable for jurisdiction-specific regulation, normative conflict resolution, and context-sensitive procedure. Agencies should require verification for AI-assisted regulatory analysis, while planning education should emphasize institutional literacy, normative judgment, and contextual sensitivity.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。