The resurgence and rapid advancement of Generative Artificial Intelligence (GenAI) in 2023 has catalyzed transformative shifts across numerous industry sectors, including urban transportation and logistics. This study investigates the evaluation of Large Language Models (LLMs), specifically GPT-4 and Phi-3-mini, to enhance transportation planning. The study assesses the performance and spatial comprehension of these models through a transportation-informed evaluation framework that includes general geospatial skills, general transportation domain skills, and real-world transportation problem-solving. Utilizing a mixed-methods approach, the research encompasses an evaluation of the LLMs' general Geographic Information System (GIS) skills, general transportation domain knowledge as well as abilities to support human decision-making in the real-world transportation planning scenarios of congestion pricing. Results indicate that GPT-4 demonstrates superior accuracy and reliability across various GIS and transportation-specific tasks compared to Phi-3-mini, highlighting its potential as a robust tool for transportation planners. Nonetheless, Phi-3-mini exhibits competence in specific analytical scenarios, suggesting its utility in resource-constrained environments. The findings underscore the transformative potential of GenAI technologies in urban transportation planning. Future work could explore the application of newer LLMs and the impact of Retrieval-Augmented Generation (RAG) techniques, on a broader set of real-world transportation planning and operations challenges, to deepen the integration of advanced AI models in transportation management practices.
TRIP-Evaluate: An Open Multimodal Benchmark for Evaluating Large Models in Transportation
School of Transportation(交通学院);Southeast University(东南大学);School of Artificial Intelligence and Computer Science(人工智能与计算机科学学院);Jiangnan University(江南大学);Department of Civil and Environmental Engineering(土木与环境工程系);Hong Kong Polytechnic University(香港理工大学)
UrbanPlanBench: A Comprehensive Urban Planning Benchmark for Evaluating Large Language Models
The Thørväld Group(Thørväld集团);Inria Paris-Rocquencourt(Inria巴黎-罗克桑court分部);Rajiv Gandhi University Doimukh(拉贾格恩迪大学多伊穆克);Tsinghua University(清华大学);Palmer Research Laboratories(帕勒默研究实验室);The Kumquat Consortium(Kumquat联盟)
PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院 Gallagher 学校);LLM Department, Hunyuan Team, Tencent(腾讯 Hunyuan 团队 LLM 部门);Beijing Academy of Artificial Intelligence(北京人工智能研究院);The Chinese University of Hong Kong(香港中文大学)
Generative AI in Transportation Planning: A Survey
Arizona State University(亚利桑那州立大学);DKS Associates(DKS咨询公司);Texas A&M University(德克萨斯A&M大学);University of Kansas(堪萨斯大学);Florida State University(佛罗里达州立大学);University of Southern California(南加州大学)
Joint Task Offloading, Inference Optimization and UAV Trajectory Planning for Generative AI Empowered Intelligent Transportation Digital Twin
Guangxi University Key Laboratory of Intelligent Networking and Scenario System (School of Information and Communication, Guilin University of Electronic Technology)(广西大学智能组网与场景系统重点实验室(桂林电子科技大学信息与通信学院));Research Institute of Highway Ministry of Transport(交通运输部公路科学研究院);School of Automation, Guangdong University of Technology(广东工业大学自动化学院);School of Architecture and Transportation Engineering, Guilin University of Electronic Technology(桂林电子科技大学建筑与交通工程学院)
Multi-Objective AI Planning: Evaluating DAE-YAHSP on a Tunable Benchmark
INRIA Saclay(法国国家信息与自动化研究所萨克雷中心);LRI(计算机研究实验室);Université Paris-Sud(巴黎南大学);ONERA-DCSD(法国航空航天实验室-DCSD);THALES Research & Technology(泰雷兹研究与技术公司)
迄今为止,所有标准的人工智能规划者都只能处理一个目标,而他们考虑多个目标的唯一方法是通过目标的聚合。此外,与单目标情况形成鲜明对比的是,不存在用于测试多目标规划算法的基准问题。DAE(Divide and Evolve)是在上届国际规划竞赛中获得单目标确定性时间满意度竞赛的进化规划方法。即使它密集地使用经典的(因此是单目标的)规划器YAHSP,也有可能将DAE-YAHSP变成多目标进化规划器。首先提出了一个可调的多目标规划基准套件,并在此基准的不同实例上比较了多目标DAE-YAHSP的几种变体的性能,希望为人工智能规划中的进一步多目标竞争铺平道路。
Re-evaluating Retrosynthesis Algorithms with Syntheseus
Microsoft Research AI for Science(微软研究院人工智能科学部门);University of Cambridge(剑桥大学);Jagiellonian University(雅盖隆大学);Johannes Kepler University Linz(林茨约翰内斯·开普勒大学)