Large language models (LLMs) are increasingly powering Text-to-SQL (Text2SQL) systems, enabling non-expert users to query industrial databases using natural language. While test-time scaling strategies have shown promise in LLM-based solutions, their effectiveness in real-world applications, especially with the latest reasoning models, remains uncertain. In this work, we benchmark six lightweight, industry-oriented test-time scaling strategies and four LLMs, including two reasoning models, evaluating their performance on the BIRD Mini-Dev benchmark. Beyond standard accuracy metrics, we also report inference latency and token consumption, providing insights relevant for practical system deployment. Our findings reveal that Divide-and-Conquer prompting and few-shot demonstrations consistently enhance performance for both general-purpose and reasoning-focused LLMs. However, introducing additional workflow steps yields mixed results, and base model selection plays a critical role. This work sheds light on the practical trade-offs between accuracy, efficiency, and complexity when deploying Text2SQL systems.
GNNs as Predictors of Agentic Workflow Performances
Shanghai Jiao Tong University(上海交通大学);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院);State Key Laboratory of General Artificial Intelligence, BIGAI(北京通用人工智能研究院通用人工智能国家重点实验室);University of Oxford(牛津大学);Peking University(北京大学)
Rethinking Retrieval: From Traditional Retrieval Augmented Generation to Agentic and Non-Vector Reasoning Systems in the Financial Domain for Large Language Models
The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check
Southeast University(东南大学);Alibaba(阿里巴巴);Southeast University Shenzhen Research Institute(东南大学深圳研究院);College of Computing and Data Science at Nanyang Technological University(南洋理工大学计算机与数据科学学院)
EvoFlow: Evolving Diverse Agentic Workflows On The Fly
The Chinese University of Hong Kong(香港中文大学);Tongji University(同济大学);Wuhan University(武汉大学);Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学);Shanghai AI Laboratory(上海人工智能实验室)
DeepWisdom(深度智慧);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Renmin University of China(中国人民大学);Nanjing University(南京大学);Fudan University(复旦大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Mila(米拉研究所);The Hong Kong University of Science and Technology(香港科技大学)
Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline
The University of Texas at Austin(德克萨斯大学奥斯汀分校);Amazon(亚马逊);Emory University(埃默里大学);Northeastern University(东北大学);Georgia Institute of Technology(佐治亚理工学院)
Design and evaluation of an agentic workflow for crisis-related synthetic tweet datasets
Department of Computer Science, University College London, London, United Kingdom(伦敦大学学院计算机科学系,伦敦,英国);National Institute of Informatics, Tokyo, Japan(日本信息学研究院,东京,日本)