Prompt optimization aims to find the best prompt to a large language model (LLM) for a given task. LLMs have been successfully used to help find and improve prompt candidates for single-step tasks. However, realistic tasks for agents are multi-step and introduce new challenges: (1) Prompt content is likely to be more extensive and complex, making it more difficult for LLMs to analyze errors, (2) the impact of an individual step is difficult to evaluate, and (3) different people may have varied preferences about task execution. While humans struggle to optimize prompts, they are good at providing feedback about LLM outputs; we therefore introduce a new LLM-driven discrete prompt optimization framework PRompt Optimization in Multi-Step Tasks (PROMST) that incorporates human-designed feedback rules to automatically offer direct suggestions for improvement. We also use an extra learned heuristic model that predicts prompt performance to efficiently sample from prompt candidates. This approach significantly outperforms both human-engineered prompts and several other prompt optimization methods across 11 representative multi-step tasks (an average 10.6\%-29.3\% improvement to current best methods on five LLMs respectively). We believe our work can serve as a benchmark for automatic prompt optimization for LLM-driven multi-step tasks. Datasets and Codes are available at https://github.com/yongchao98/PROMST. Project Page is available at https://yongchao98.github.io/MIT-REALM-PROMST.
Heuristic Classification of Thoughts Prompting (HCoT): Integrating Expert System Heuristics for Structured Reasoning into Large Language Models
Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；Shenyang Aerospace University（沈阳航空航天大学）；ZGC LAB, Beijing,China（北京ZGC实验室）；Institute for Network Sciences and Cyberspace, Tsinghua University（清华大学网络科学与网络空间研究院）
Bridging Multi-Valued Heuristics and Dimensionality Reduction in Multi-Objective Search
Department of Computer Science, Technion – Israel Institute of Technology（技术学院计算机科学系，以色列理工学院）；Faculty of Computer and Information Science, Ben-Gurion University of the Negev（贝内-格尔离子大学内盖夫分校计算机与信息科学学院）
Heuristics and Biases in AI Decision-Making: Implications for Responsible AGI
Golisano Institute for Sustainability Rochester Institute of Technology Rochester, The United States；Information Sciences \& Technology, CEHC University at Albany, SUNY Albany, The United States
One Size Fits None: Heuristic Collapse in LLM Investment Advice
Computer Science and Artificial Intelligence Laboratory（计算机科学与人工智能实验室）；Massachusetts Institute of Technology（麻省理工学院）；Sloan School of Management（斯隆管理学院）
Role-aware Heuristic Episodic Attention for Conversational LLMs
National Key Laboratory of Parallel and Distributed Computing（平行与分布式计算国家重点实验室）；College of Computer Science and Technology（计算机科学与技术学院）；National University of Defense Technology（国防科学技术大学）
REMoH: A Reflective Evolution of Multi-objective Heuristics approach via Large Language Models
Vicomtech Foundation, Basque Research and Technology Alliance (BRTA)（维克姆技基金会、巴斯克研究与技术联盟（BRTA））；Department of Computer Sciences and Artificial Intelligence, University of the Basque Country (UPV/EHU)（巴斯克国家大学（UPV/EHU）计算机科学与人工智能系）；School of Applied Sciences and Engineering, Universidad EAFIT（埃法伊特大学应用科学与工程学院）；HiTZ Basque Center for Language Technology, University of the Basque Country (UPV/EHU)（巴斯克国家大学（UPV/EHU）HiTZ巴斯克语言技术中心）
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering
School of Information Science and Technology, Hangzhou Normal University（杭州师范大学信息科学与技术学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）
Integrating Artificial Intelligence into Operating Systems: A Survey on Techniques, Applications, and Future Directions
School of Software Technology, Zhejiang University（浙江大学软件学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Mathematical Engineering and Advanced Computing（数学工程与先进计算国家重点实验室）；Chinese Academy of Engineering（中国工程院）；College of Cyber Security, Jinan University（暨南大学网络安全学院）
