在跨语言潜在空间中对多语言大语言模型进行去偏置
Debiasing Multilingual LLMs in Cross-lingual Latent Space
AI总结：
针对多语言大语言模型去偏置技术跨语言迁移性有限的问题，本研究提出在由平行TED脚本训练的自动编码器构建的对齐跨语言潜在空间中进行去偏置，实验证明该方法能显著提升去偏置性能及跨语言迁移性。
摘要：
SentDebias等去偏置技术旨在减少大语言模型（LLMs）中的偏差。此前研究通过直接将这些方法应用于LLM表示来评估其跨语言迁移性，发现其跨语言有效性有限。因此，本研究提出在联合潜在空间而非直接在LLM表示上进行去偏置。我们使用在平行TED演讲脚本上训练的自动编码器构建对齐良好的跨语言潜在空间。针对Aya-expanse模型和两种去偏置技术在四种语言（英语、法语、德语、荷兰语）上的实验表明：a）自动编码器能有效构建对齐良好的跨语言潜在空间；b）在学习到的跨语言潜在空间中应用去偏置技术，显著提升了整体去偏置性能和跨语言迁移性。
英文摘要：
Debiasing techniques such as SentDebias aim to reduce bias in large language models (LLMs). Previous studies have evaluated their cross-lingual transferability by directly applying these methods to LLM representations, revealing their limited effectiveness across languages. In this work, we therefore propose to perform debiasing in a joint latent space rather than directly on LLM representations. We construct a well-aligned cross-lingual latent space using an autoencoder trained on parallel TED talk scripts. Our experiments with Aya-expanse and two debiasing techniques across four languages (English, French, German, Dutch) demonstrate that a) autoencoders effectively construct a well-aligned cross-lingual latent space, and b) applying debiasing techniques in the learned cross-lingual latent space significantly improves both the overall debiasing performance and cross-lingual transferability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
