对LLM进行条件训练以生成混合语言文本
Conditioning LLMs to Generate Code-Switched Text
AI总结：
本研究通过回译和微调LLM，探索其生成混合语言文本的能力，发现LLM基于判断更符合人类偏好，拓展了代码切换研究的前景。
摘要：
代码切换（CS）仍然是自然语言处理（NLP）中的关键挑战，由于大规模、多样化的CS数据集有限，难以进行稳健的训练和评估。尽管近年来有进展，但LLM在处理CS方面的能力和限制仍不完全清楚。在本工作中，我们研究了LLM在代码切换文本生成框架中的应用程度，重点是英语-西班牙语语言对。我们提出的方法包括将自然CS句子回译成单语英语，并利用由此产生的平行语料库对LLM进行微调，以将单语句子转换为CS文本。我们通过人类偏好研究、定性错误分析、使用流行参考基度量和LLM基于判断的评估彻底分析了模型的性能。结果显示，微调是确保当前LLM能够一致生成流畅的代码切换文本的关键步骤，且我们的方法生成高质量的输出，拓展了CS通信的研究机会。我们发现传统度量标准在评估生成的CS数据质量时与人类判断不相关，但LLM基于判断更接近人类偏好。我们以CC-BY-NC-SA许可发布了我们的代码和生成的数据集。
英文摘要：
Code-switching (CS) is still a critical challenge in Natural Language Processing (NLP), due to the limited availability of large-scale, diverse CS datasets for robust training and evaluation. Despite recent advances, the capabilities and limitations of LLMs in handling CS are still not fully understood. In this work, we investigate the extent to which LLMs can be used in a framework for CS text generation, focusing on the English-Spanish language pair. Our proposed methodology consists of back-translating natural CS sentences into monolingual English, and using the resulting parallel corpus to fine-tune LLMs to turn monolingual sentences into CS. We thoroughly analyse the models' performance through a study on human preferences, a qualitative error analysis, an evaluation with popular reference-based metrics and LLM-based judgment. Results show that fine-tuning can be a key step to ensure that current LLMs consistently generate fluent code-switched text and that our methodology generates high-quality outputs, expanding research opportunities in CS communication. We find that traditional metrics do not correlate with human judgement when assessing the quality of the generated CS data, but LLM-based judgment aligns more closely with human preferences. We release our code and generated dataset under a CC-BY-NC-SA license.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
