Parameter-efficient fine-tuning (PEFT) methods are increasingly vital in adapting large-scale pre-trained language models for diverse tasks, offering a balance between adaptability and computational efficiency. They are important in Low-Resource Language (LRL) Neural Machine Translation (NMT) to enhance translation accuracy with minimal resources. However, their practical effectiveness varies significantly across different languages. We conducted comprehensive empirical experiments with varying LRL domains and sizes to evaluate the performance of 8 PEFT methods with in total of 15 architectures using the SacreBLEU score. We showed that 6 PEFT architectures outperform the baseline for both in-domain and out-domain tests and the Houlsby+Inversion adapter has the best performance overall, proving the effectiveness of PEFT methods.
Unlocking the Potential of Model Merging for Low-Resource Languages
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);State Key Laboratory of General Artificial Intelligence, Peking University(北京大学通用人工智能国家重点实验室);Center for Data Science, Peking University(北京大学数据科学中心);College of Engineering, Peking University(北京大学工学院)
CALM Before the STORM: Unlocking Native Reasoning for Optimization Modeling
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Qwen Team, Alibaba Inc.(阿里巴巴集团文勤团队);Shanghai University of Finance and Economics(上海财经大学);Southern University of Science and Technology(南方科技大学);Shenzhen Loop Area Institute (SLAI)(深圳环湖研究院)
PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning
Nanjing University(南京大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Southern University of Science and Technology(南方科技大学);Shanghai University of Finance and Economics(上海财经大学)
Unlocking Fine-Grained Translation Quality Estimation in LRMs through Mutually Boosting Implicit and Explicit Reasoning
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室);Huawei Translation Services Center, Beijing, China(北京华为翻译服务中心)
From Completion to Editing: Unlocking Context-Aware Code Infilling via Search-and-Replace Instruction Tuning
USTC(University of Science and Technology of China);Alibaba Group(阿里巴巴集团);CASIA(Chinese Academy of Sciences Institute of Automation);SIAT(Institute of Automation, Chinese Academy of Sciences)
INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院);School of Artificial Intelligence, Nankai University(南开大学人工智能学院)
Unlocking the Potential of ChatGPT: A Comprehensive Exploration of its Applications, Advantages, Limitations, and Future Directions in Natural Language Processing
MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation
School of Computer Science and Technology, Tianjin University, Tianjin, China(天津大学计算机科学与技术学院);School of Software, Tsinghua University, Beijing, China(清华大学软件学院);School of Information Resource Management, Renmin University of China,Beijing, China(中国人民大学信息资源管理学院);School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China(上海交通大学人工智能学院);Baidu Inc., Beijing, China(百度公司)
City University of Hong Kong(香港城市大学);Rutgers University(罗格斯大学);University of Science and Technology of China(中国科学技术大学);Universiteit van Amsterdam(阿姆斯特丹大学);Baidu Inc(百度公司)
Beyond Vanilla Fine-Tuning: Leveraging Multistage, Multilingual, and Domain-Specific Methods for Low-Resource Machine Translation
University of Moratuwa(莫拉图瓦大学);Saarland University(萨尔大学);University of Toronto(多伦多大学);University of Ontario of Technology(安大略理工大学);Massey University(梅西大学)
Attentive fine-tuning of Transformers for Translation of low-resourced languages @LoResMT 2021
Indian Institute of Information Technology Tiruchirappalli(印度特里奇拉帕利信息技术学院);ULTRA Arts and Science College(ULTRA文理学院);Sri Sivasubramaniya Nadar College of Engineering(斯里·西瓦苏布拉马尼亚·纳达尔工程学院);Kongu Engineering College(孔古工程学院);Sultan Idris Education University(苏丹依德里斯教育大学);National University of Ireland Galway(爱尔兰国立大学戈尔韦分校)
本文报告了IIITT团队为英语-马拉地语和英语-爱尔兰语语言对LoResMT 2021共享任务提交的机器翻译(MT)系统。该任务的重点是为爱尔兰语和马拉地语等资源相对较低的语言提供出色的翻译。我们微调IndicTrans,这是一个针对英语->马拉地语的预训练多语言NMT模型,使用外部并行语料库作为额外训练的输入。我们对后一种语言对使用了预先训练的Helsinki-NLP Opus MT English->Irish模型。我们的方法在BLEU指标上产生了相对有希望的结果。在团队名称IIITT下,我们的系统分别在英语->马拉地语,爱尔兰语->英语和英语->爱尔兰语中排名1,1和2。