Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis
Artefact Research Center(Artefact 研究中心);Unbabel;Equall;MICS, CentraleSupélec, Université Paris-Saclay(巴黎-萨克雷大学中央理工-高等电力学院数学与计算机科学实验室);Instituto de Telecomunicações(电信研究所);Instituto Superior Técnico & Universidade de Lisboa(里斯本大学高等技术学院)
Hippolyte Gisserot-Boukhlef, Ricardo Rei, Emmanuel Malherbe, Céline Hudelot, Pierre Colombo, Nuno M. Guerreiro
Neural metrics for machine translation (MT) evaluation have become increasingly prominent due to their superior correlation with human judgments compared to traditional lexical metrics. Researchers have therefore utilized neural metrics through quality-informed decoding strategies, achieving better results than likelihood-based methods. With the rise of Large Language Models (LLMs), preference-based alignment techniques have gained attention for their potential to enhance translation quality by optimizing model weights directly on preferences induced by quality estimators. This study focuses on Contrastive Preference Optimization (CPO) and conducts extensive experiments to evaluate the impact of preference-based alignment on translation quality. Our findings indicate that while CPO consistently outperforms Supervised Fine-Tuning (SFT) on high-quality data with regard to the alignment metric, it may lead to instability across downstream evaluation metrics, particularly between neural and lexical ones. Additionally, we demonstrate that relying solely on the base model for generating candidate translations achieves performance comparable to using multiple external systems, while ensuring better consistency across downstream metrics.
From Utterance to Vividity: Training Expressive Subtitle Translation LLM via Adaptive Local Preference Optimization
MAIS, Institute of Automation, Chinese Academy of Sciences(自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Hujing Digital Media & Entertainment Group(华景数字媒体与娱乐集团);Geely AI lab(吉利AI实验室)
Decisive: Guiding User Decisions with Optimal Preference Elicitation from Unstructured Documents
Adobe Research, India(Adobe印度研究实验室);IIT Madras(印度理工学院马德拉斯分校);IIT Roorkee(印度理工学院罗尔基分校);IIIT Hyderabad(Hyderabad印度理工学院);University of Maryland, College Park(马里兰大学学院公园分校)
Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization
Alibaba Digital Media and Entertainment Group(阿里巴巴数字媒体与娱乐集团);School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件学院);Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University(北京交通大学交通数据挖掘与具身智能重点实验室)
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation
Instituto de Telecomunicações(电信研究所);Unbabel(Unbabel公司);Instituto Superior Técnico, Universidade de Lisboa(里斯本大学高等技术学院);Carnegie Mellon University(卡内基梅隆大学);CentraleSupélec, Université Paris-Saclay(巴黎萨克雷大学中央高等电力学院);ELLIS Unit Lisbon(里斯本ELLIS单元)
Cross-Preference Learning for Sentence-Level and Context-Aware Machine Translation
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);School of Computer Science and Artificial Intelligence, Zhengzhou University(郑州大学计算机与人工智能学院);Huawei Translation Services Center(华为翻译服务中心)
Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solution
School of Computer Science, Fudan University(复旦大学计算机科学学院);College of Foreign Languages and Literature, Fudan University(复旦大学外国语言文学学院);Institute of Modern Languages and Linguistics, Fudan University(复旦大学现代语言与语言学研究所)
SimulPL: Aligning Human Preferences in Simultaneous Machine Translation
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Graduate School of Translation and Interpretation, Beijing Foreign Studies University(北京外国语大学高级翻译学院)
Vocabulary Manipulation for Neural Machine Translation
T.J. Watson Research Center(托马斯·J·沃森研究中心);IBM
为了捕捉丰富的语言现象,神经机器翻译模型必须使用大的词汇量,这需要高计算时间和大内存使用。在本文中,我们缓解这个问题,通过引入一个并行级或批量级词汇,这只是一个非常小的子集的完整输出词汇。对于每一个句子或批次,我们只预测其词汇水平或批次水平词汇中的目标词。因此,我们减少了计算时间和内存的使用。我们的方法简单地考虑了源句子中每个单词或短语的翻译选项,并基于从传统机器翻译模型中学习的单词到单词的翻译模型或双语短语库为每个句子选择一个非常小的目标词汇。在大规模英法翻译任务上的实验结果表明,我们的方法比Jean et al.(2015)的大词汇量神经机器翻译系统的翻译性能好1个BLEU点。