Enhancing the Non-Functional Quality Compliance of LLM-Generated Code through Quality-Aware Preference Learning
School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院);School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算与信息系统学院)
Liang Lu, Yuan Jiang, Christoph Treude, Jingyu Xiao, Xiaohong Su, Michael R. Lyu
Large Language Models (LLMs) have been widely adopted in commercial code completion engines, significantly enhancing coding efficiency and productivity. However, even functionally correct LLM-generated code may exhibit non-functional quality issues that violate coding standards and best practices, such as poor style and limited maintainability. To address this, we propose a framework for quality-aware preference learning that guides LLMs toward generating criteria-compliant code. Our approach consists of three phases. First, we construct a dataset of paired criteria-violating and criteria-compliant samples, where each pair contains code exhibiting a specific non-functional quality issue and its repaired version that resolves the issue. Second, we design an adaptive token weighting mechanism to emphasize quality-sensitive code regions. Third, we introduce a hybrid optimization objective that combines ranking loss with language modeling loss and KL divergence to enable effective comparative optimization. Extensive experiments on DeepSeek-Coder and Qwen2.5-Coder show that our method substantially improves compliance with the targeted non-functional quality criteria while maintaining functional correctness, achieving a 75.7% relative increase in Quality Reciprocal Score (QRS) on MBPP-sanitized for Qwen2.5-Coder. Fine-tuning a 7B model requires less than three hours, indicating strong practical viability. Ablation studies and a user study further support the effectiveness of the proposed framework.
FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
LMU Munich(慕尼黑大学);Columbia University(哥伦比亚大学);University of Southern California(南加州大学);University of Oxford(牛津大学);Munich Center for Machine Learning(慕尼黑机器学习中心)
针对 DPO 难以纠正错误排序偏好对的问题,本文提出 FocalPO,通过 Focal Loss 式调制因子动态降权错误样本、强化正确样本,并在多个基准和模型上优于 DPO 变体。
Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization
Technische Universität Berlin(柏林技术大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);University of Duisburg-Essen(杜伊斯堡- Essen大学);LMU Munich(慕尼黑大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Saarland Informatics Campus(萨尔兰州信息学校区);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究院);Centre for European Research in Trusted AI (CERTAIN)(可信人工智能欧洲研究中心)
Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization
Capital One(第一资本);Northeastern University(东北大学);Apple Inc.(苹果公司)
AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimization
School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);State Key Laboratory of Cognitive Intelligence(认知智能国家重点实验室);Tongyi Lab(通义实验室)
Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples
University of Maryland, College Park(马里兰大学 College Park 分校);IIT Bombay(印度理工学院班加罗尔分校);IIT Patna(印度理工学院帕坦分校);Adobe Research(Adobe 研究院);IIT Kanpur(印度理工学院坎普尔分校)
University of Arizona(亚利桑那大学);Avirtek, Inc.(Avirtek公司);Joint Interoperability Test Command (JITC), United States Department of Defense(美国国防部联合互操作性测试司令部)
Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
Shandong University(山东大学);Southern University of Science and Technology(南方科技大学);University of Georgia(佐治亚大学);National University of Singapore(新加坡国立大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
From COBIT to ISO 42001: Evaluating Cybersecurity Frameworks for Opportunities, Risks, and Regulatory Compliance in Commercializing Large Language Models
Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimization
Shandong University(山东大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);New York University(纽约大学);Xi'an Jiaotong University(西安交通大学);Australian National University(澳大利亚国立大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)