优化建模与LLM的综述:进展与未来方向
A Survey of Optimization Modeling Meets LLMs: Progress and Future Directions
AI总结:
本综述回顾了LLM在优化建模技术栈中的最新进展,分析了基准数据集的高错误率并构建了公平评估的新排行榜,同时指出了当前局限与未来方向。
摘要:
凭借在解决现实世界问题中的巨大实用性,优化建模已被广泛应用于各行各业的最佳决策制定中,但它需要运筹学专业人员的丰富专业知识。随着大型语言模型(LLMs)的出现,自动化数学建模过程迎来了新的机遇。本综述对涵盖整个技术栈的最新进展进行了全面及时的回顾,包括基础模型的数据合成与微调、推理框架、基准数据集和性能评估。此外,我们对基准数据集的质量进行了深入分析,发现其错误率高得惊人。我们清理了数据集,并围绕基础LLM模型和数据集构建了一个具有公平性能评估的新排行榜。我们还建立了一个在线门户,整合了清理后的数据集、代码和论文资源库,以造福社区。最后,我们指出了当前方法的局限性,并概述了未来的研究机会。
英文摘要:
By virtue of its great utility in solving real-world problems, optimization modeling has been widely employed for optimal decision-making across various sectors, but it requires substantial expertise from operations research professionals. With the advent of large language models (LLMs), new opportunities have emerged to automate the procedure of mathematical modeling. This survey presents a comprehensive and timely review of recent advancements that cover the entire technical stack, including data synthesis and fine-tuning for the base model, inference frameworks, benchmark datasets, and performance evaluation. In addition, we conducted an in-depth analysis on the quality of benchmark datasets, which was found to have a surprisingly high error rate. We cleaned the datasets and constructed a new leaderboard with fair performance evaluation in terms of base LLM model and datasets. We also build an online portal that integrates resources of cleaned datasets, code and paper repository to benefit the community. Finally, we identify limitations in current methodologies and outline future research opportunities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。