arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.14516cs.AIcs.CLcs.IR
收藏

超越文字:在交通规划中评估大型语言模型

Beyond Words: Evaluating Large Language Models in Transportation Planning

The Pennsylvania State University(宾夕法尼亚州立大学);NCS Group(NCS集团)

Shaowei Ying, Zhenlong Li, Manzhu Yu

AI总结:

本研究通过交通信息评估框架评估GPT-4和Phi-3-mini在交通规划中的性能,发现GPT-4在GIS和交通任务中表现更优,Phi-3-mini适用于资源受限场景,凸显GenAI变革城市交通规划的潜力。

摘要:

2023年生成式人工智能(GenAI)的复兴与快速发展催化了众多行业领域的变革,包括城市交通与物流。本研究探讨了大型语言模型(LLMs)的评估,特别是GPT-4和Phi-3-mini,以增强交通规划。该研究通过一个包含通用地理空间技能、通用交通领域技能和现实世界交通问题解决的交通信息评估框架,评估了这些模型的性能和空间理解能力。利用混合方法,研究涵盖了对LLMs通用地理信息系统(GIS)技能、通用交通领域知识的评估,以及在拥堵定价的现实交通规划场景中支持人类决策的能力。结果表明,与Phi-3-mini相比,GPT-4在各种GIS和特定交通任务中展现出更高的准确性和可靠性,凸显了其作为交通规划者稳健工具的潜力。尽管如此,Phi-3-mini在特定分析场景中表现出能力,表明其在资源受限环境中的实用性。研究结果强调了GenAI技术在城市交通规划中的变革潜力。未来工作可探索更新的LLMs应用及检索增强生成(RAG)技术对更广泛现实交通规划与运营挑战的影响,以深化先进AI模型在交通管理实践中的整合。

英文摘要:

The resurgence and rapid advancement of Generative Artificial Intelligence (GenAI) in 2023 has catalyzed transformative shifts across numerous industry sectors, including urban transportation and logistics. This study investigates the evaluation of Large Language Models (LLMs), specifically GPT-4 and Phi-3-mini, to enhance transportation planning. The study assesses the performance and spatial comprehension of these models through a transportation-informed evaluation framework that includes general geospatial skills, general transportation domain skills, and real-world transportation problem-solving. Utilizing a mixed-methods approach, the research encompasses an evaluation of the LLMs' general Geographic Information System (GIS) skills, general transportation domain knowledge as well as abilities to support human decision-making in the real-world transportation planning scenarios of congestion pricing. Results indicate that GPT-4 demonstrates superior accuracy and reliability across various GIS and transportation-specific tasks compared to Phi-3-mini, highlighting its potential as a robust tool for transportation planners. Nonetheless, Phi-3-mini exhibits competence in specific analytical scenarios, suggesting its utility in resource-constrained environments. The findings underscore the transformative potential of GenAI technologies in urban transportation planning. Future work could explore the application of newer LLMs and the impact of Retrieval-Augmented Generation (RAG) techniques, on a broader set of real-world transportation planning and operations challenges, to deepen the integration of advanced AI models in transportation management practices.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑