The alignment of large language models (LLMs) with human values is critical as these models become increasingly integrated into various societal and decision-making processes. Traditional methods, such as reinforcement learning from human feedback (RLHF), achieve alignment by fine-tuning model parameters, but these approaches are often computationally expensive and impractical when models are frozen or inaccessible for parameter modification. In contrast, prompt optimization is a viable alternative to RLHF for LLM alignment. While the existing literature has shown empirical promise of prompt optimization, its theoretical underpinning remains under-explored. We address this gap by formulating prompt optimization as an optimization problem and try to provide theoretical insights into the optimality of such a framework. To analyze the performance of the prompt optimization, we study theoretical suboptimality bounds and provide insights in terms of how prompt optimization depends upon the given prompter and target model. We also provide empirical validation through experiments on various datasets, demonstrating that prompt optimization can effectively align LLMs, even when parameter fine-tuning is not feasible.
Hanoi University of Science and Technology（河内科学技术大学）；Deakin University（德金大学）；Max Planck Research School for Intelligent Systems（马克斯·普朗克智能系统研究学校）；VinUniversity（文大学）；Monash University（墨尔本大学）
PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibration
University of Queensland（昆士兰大学）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）；Provable Responsible AI and Data Analytics (PRADA) Lab（可证责任AI与数据分析师实验室）；King Abdullah University of Science and Technology (KAUST)（国王 Abdullah 科学与技术大学）
Differentially Private Preference Data Synthesis for Large Language Model Alignment
Department of Computer Science, University of Virginia, Charlottesville, Virginia, USA（弗吉尼亚大学计算机科学系）；Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, Virginia, USA（弗吉尼亚大学电气与计算机工程系）
PromptGuard: An Orchestrated Prompting Framework for Principled Synthetic Text Generation for Vulnerable Populations using LLMs with Enhanced Safety, Fairness, and Controllability
Posts and Telecommunications Institute of Technology（邮电技术研究所）；Hanoi Architectural University（河内建筑大学）
TASU2: Controllable CTC Simulation for Alignment and Low-Resource Adaptation of Speech LLMs
X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系X-LANCE实验室）；MoE Key Lab of Artificial Intelligence（教育部人工智能重点实验室）；Jiangsu Key Lab of Language Computing（江苏省语言计算重点实验室）；AISpeech Ltd（思必驰科技股份有限公司）；Nanjing University（南京大学）
PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；Data Science & Artificial Intelligence Research Institute, China Unicom（中国unicom数据科学与人工智能研究院）；Unicom Data Intelligence, China Unicom（中国unicom数据智能）
From Sycophantic Consensus to Pluralistic Repair: Why AI Alignment Must Surface Disagreement
Department of Computer Science, University of Oxford（牛津大学计算机科学系）；Institute for Ethics in AI, University of Oxford（牛津大学人工智能伦理研究所）；Responsible Technology Institute, University of Oxford（牛津大学负责任技术研究所）
A Statistical Case Against Empirical Human-AI Alignment
Department of Statistics, LMU Munich（统计系，慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Research Group Neuroinformatics, Faculty of Computer Science, University of Vienna（神经信息学研究组，维也纳大学）；Doctoral School Computer Science, Faculty of Computer Science, University of Vienna（计算机科学博士学院，维也纳大学）；School of Computing & Communications, Lancaster University Leipzig（计算与通信学院，莱比锡 Lancaster 大学）
Assessing Alignment and Stability of Feature Importance Explanations via Weight of Evidence
SMARTEST Research Center（SMARTEST研究中心）；eCampus University（eCampus大学）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）；University of Florence（佛罗伦萨大学）；University of Pisa（比萨大学）
Scalable Valuation of Human Feedback through Provably Robust Model Alignment
The University of Osaka（大阪大学）；Lattice Lab, Toyota Motor Corporation（丰田公司Lattice实验室）；Machine Learning Research Group, University of Oxford（牛津大学机器学习研究组）；RIKEN AIP（理化学研究所AIP）
