Prompt quality plays a central role in controlling the behavior, reliability, and reasoning performance of large language models (LLMs), particularly for smaller open-source instruction-tuned models that depend heavily on explicit structure. While recent work has explored automatic prompt optimization using textual gradients and self-refinement, most existing methods treat prompts as monolithic blocks of text, making it difficult to localize errors, preserve critical instructions, or prevent uncontrolled prompt growth. We introduce Modular Prompt Optimization (MPO), a schema-based prompt optimization framework that treats prompts as structured objects composed of fixed semantic sections, including system role, context, task description, constraints, and output format. MPO applies section-local textual gradients, generated by a critic language model, to refine each section independently while keeping the overall prompt schema fixed. Section updates are consolidated through de-duplication to reduce redundancy and interference between components, yielding an interpretable and robust optimization process. We evaluate MPO on two reasoning benchmarks, ARC-Challenge and MMLU, using LLaMA-3 8B-Instruct and Mistral-7B-Instruct as solver models. Across both benchmarks and models, MPO consistently outperforms an untuned structured prompt and the TextGrad baseline, achieving substantial accuracy gains without modifying model parameters or altering prompt structure. These results demonstrate that maintaining a fixed prompt schema while applying localized, section-wise optimization is an effective and practical approach for improving reasoning performance in small open-source LMs.
AutoMedPrompt: A New Framework for Optimizing LLM Medical Prompts Using Textual Gradients
Keck Data Science Institute, Pepperdine University（佩珀代因大学凯克数据科学研究所）；Department of Medicine, University of California, Los Angeles（加利福尼亚大学洛杉矶分校医学院）；Biological Sciences Division, University of Chicago（芝加哥大学生物科学部）；Brain Research Institute, University of California, Los Angeles（加利福尼亚大学洛杉矶分校脑研究所）
REVOLVE: Optimizing AI Systems by Tracking Response Evolution in Textual Optimization
Hong Kong University of Science and Technology（香港科技大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Brown University（布朗大学）；University of Michigan - Ann Arbor（密歇根大学安娜堡分校）；Hong Kong Polytechnic University（香港理工大学）；Intel Lab（英特尔实验室）
From Weak Task Specifications to Scientific Extraction Agents: Optimizing Task Construction
University of Science and Technology of China（中国科学技术大学）；State Key Laboratory of Cognitive Intelligence（认知智能全国重点实验室）；CCCC Second Highway Consultants Co., Ltd.（中交第二公路勘察设计研究院有限公司）
Towards Modular LLMs by Building and Reusing a Library of LoRAs
Microsoft Research（微软研究院）；Mila — Quebec AI Institute（米拉魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）；University of Copenhagen（哥本哈根大学）；University of Edinburgh（爱丁堡大学）；HEC Montréal（蒙特利尔高等商学院）
Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning
Sun Yat-sen University（中山大学）；Shenzhen Loop Area Institute（深圳河套学院）；Beijing Zhongguancun Academy（北京中关村研究院）；Huawei Technologies Co., Ltd（华为技术有限公司）；Harbin Institute of Technology（哈尔滨工业大学）
A Similarity Paradigm Through Textual Regularization Without Forgetting
Shanghai Jiao Tong University（上海交通大学）；Hong Kong Baptist University（香港浸会大学）；Northeastern University（东北大学）；University of Science and Technology of China（中国科学技术大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
