Modern language models demonstrate impressive coding capabilities in common programming languages (PLs), such as C++ and Python, but their performance in lower-resource PLs is often limited by training data availability. In principle, however, most programming skills are universal across PLs, so the capability acquired in one PL should transfer to others. In this work, we propose the task of zero-shot cross-programming-language transfer for code RL. We find that, for Llama-3.1, RL training for code generation in a source PL fails to improve, and sometimes even degrades, the performance on other target PLs. To address this, we hypothesize that effective RL transfer requires a generalizable SFT initialization before RL. We thus propose **Parallel-SFT**, an SFT strategy that incorporates "parallel programs" -- functionally equivalent code implemented in multiple PLs -- into the data mixture. We demonstrate that this improves transferability: when we subsequently perform RL on our Parallel-SFT model, we observe better generalization to unseen PLs. Analysis of the model internal representations reveals that Parallel-SFT leads to a more functionality-centric latent space, where equivalent programs across PLs are more tightly clustered, which we hypothesize to contribute to the improved transferability.
MPL: Multiple Programming Languages with Large Language Models for Information Extraction
State Key Laboratory of Intelligent Power Distribution Equipment and System(智能电力设备与系统国家重点实验室);School of Health Sciences and Biomedical Engineering(健康科学与生物医学工程学院);National Engineering Research Center for Software Engineering(软件工程国家工程研究中心);Peking University(北京大学);School of Artificial Intelligence(人工智能学院)
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Institute for Clarity in Documentation(文档清晰性研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗克库尔中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
Large Language Models for Anomaly Detection in Computational Workflows: from Supervised Fine-Tuning to In-Context Learning
Mathematics and Computer Science Division, Argonne National Laboratory(阿贡国家实验室数学与计算机科学部);Information Sciences Institute, University of Southern California(南加州大学信息科学研究所);RENCI, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校RENCI)
Tourism Question Answer System in Indian Language using Domain-Adapted Foundation Models
Department of Linguistics Banaras Hindu University(语言学系班加罗尔 Hindu 大学);Department of Humanistic Studies Indian Institute of Technology (BHU)(人文科学系印度理工学院(BHU));Department of Computer Science and Engineering Indian Institute of Technology Bhilai(计算机科学与工程系印度理工学院 Bhilai)
An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
University of Washington(华盛顿大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Stanford University(斯坦福大学);Georgia Institute of Technology(佐治亚理工学院);University of California, Riverside(加州大学河滨分校);Microsoft Research NYC(微软研究院纽约分部)