Direct Preference Optimization (DPO) is widely used after supervised fine-tuning (SFT) to align language models, yet empirical behavior under small backbones and modest data is under-specified. We systematically compare SFT-only, DPO-only, and staged SFT-to-DPO training alongside full fine-tuning (FFT) versus LoRA on a GPT-2-scale decoder, evaluating paraphrase detection and Shakespearean sonnet continuation. DPO yields small, task-dependent gains over strong SFT and can match competitive SFT accuracy without a warm start when the preference construction closely parallels the supervised objective. In contrast, parameterization dominates: FFT consistently outperforms LoRA at matched training depth, and LoRA does not reduce wall-clock time on our hardware. These findings indicate that, in this small-scale regime, supervised full-parameter adaptation remains the primary performance lever, while preference optimization and low-rank adaptation provide limited marginal returns.
Learning When to Reason for Text-to-SQL via SFT and DPO
Department of Electrical and Computer Engineering, Seoul National University（首尔国立大学电气与计算机工程系）；AI Center, Samsung Electronics（三星电子人工智能中心）；AIIS, ASRI, INMC, ISRC, and IPAI, Seoul National University（首尔国立大学人工智能研究所、高级科学研究所以及综合纳米技术中心、信息存储研究中心和人工智能平台研究所）
Efficient Training of Robust Traditional Chinese LLaMA-1B on a Single Consumer GPU: Continual Pre-training, SFT, and DPO
National Tsing Hua University（台湾清华大学）；Accton Technology Corporation（智邦科技公司）；National Yang Ming Chiao Tung University（阳明交通大学）；University of Taipei（台北市立大学）
Northeastern Uni at Multilingual Counterspeech Generation: Enhancing Counter Speech Generation with LLM Alignment through Direct Preference Optimization
Capital One（第一资本）；Northeastern University（东北大学）；Apple Inc.（苹果公司）
RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Qwen LLM Application Team, Alibaba（阿里巴巴Qwen大模型应用团队）；Stanford University（斯坦福大学）
Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
National Laboratory for Scientific Computing (LNCC)（国家科学计算实验室）；Institute of Artificial Intelligence（人工智能研究所）
Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities
University of Zurich（苏黎世大学）；EPFL（瑞士联邦理工学院）；Zurich University of Applied Sciences（苏黎世应用科学大学）
Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization
School of Computing and Intelligent Innovation, Fudan University（复旦大学计算与智能创新学院）；School of Economics and Management, Tongji University（同济大学经济与管理学院）；College of Information Technology, Shanghai Ocean University（上海海洋大学信息学院）
DavIR: Data Selection via Implicit Reward for Large Language Models
ByteDance, Inc.（字节跳动公司）；Cold Spring Harbor Laboratory（冷泉港实验室）；Shanghai Jiao Tong University（上海交通大学）；National University of Singapore（新加坡国立大学）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）
From Oracle to Noisy Context: Mitigating Contextual Exposure Bias in Speech-LLMs
School of Computer Science and Technology, Xinjiang University, Urumqi, China（新疆大学计算机科学与技术学院，乌鲁木齐，中国）；Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Urumqi, China（丝绸之路多语种认知计算联合国际研究实验室，乌鲁木齐，中国）；Timekettle
Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection
School of Life Sciences, IDG/McGovern Institute for Brain Research（生命科学学院，IDG/麦戈文脑科学研究所）；Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center（计算机科学与技术系，人工智能研究所，清华大学-博世联合机器学习中心，THBI实验室，BNRist中心）；Tsinghua University, Beijing, China（清华大学，北京，中国）
Weights-Rotated Preference Optimization for Large Language Models
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Baidu Inc.（百度公司）
Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model
Harvard T.H. Chan School of Public Health（哈佛陈曾熙公共卫生学院）；Peking University（北京大学）；University of Oxford（牛津大学）；Alibaba Group（阿里巴巴集团）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Ant Group（蚂蚁集团）；Tianyi Traffic Technology（天翼交通科技）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
ShanghaiTech University（上海科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Meituan（美团）；Alibaba Group（阿里巴巴集团）；The Hong Kong University of Science and Technology（香港科技大学）
