ERUPD -- 英语至罗马乌尔都语平行数据集
ERUPD -- English to Roman Urdu Parallel Dataset
AI总结:
本研究构建了包含75,146个句子对的英语至罗马乌尔都语平行数据集ERUPD,通过结合高级提示工程生成的合成数据与真实对话数据,并经人工评估完善,为机器翻译等任务提供重要资源。
摘要:
弥合语言鸿沟能促进全球发展与文化交流。本研究通过创建一个包含75,146个句子对的新型平行数据集,解决了罗马乌尔都语(即乌尔都语的拉丁字母变体,在数字通信中被广泛使用)所面临的挑战。罗马乌尔都语缺乏标准化、语音变异性以及与英语的语码转换,使其语言处理变得复杂。我们通过采用一种混合方法来解决这一问题,该方法将通过高级提示工程生成的合成数据与来自个人消息群组的真实对话数据相结合。我们进一步通过人工评估阶段完善了该数据集,解决了语言不一致的问题,并确保了语码转换、语音表示和同义变异性方面的准确性。最终生成的数据集捕捉了罗马乌尔都语多样的语言特征,可作为机器翻译、情感分析和多语言教育的重要资源。
英文摘要:
Bridging linguistic gaps fosters global growth and cultural exchange. This study addresses the challenges of Roman Urdu -- a Latin-script adaptation of Urdu widely used in digital communication -- by creating a novel parallel dataset comprising 75,146 sentence pairs. Roman Urdu's lack of standardization, phonetic variability, and code-switching with English complicates language processing. We tackled this by employing a hybrid approach that combines synthetic data generated via advanced prompt engineering with real-world conversational data from personal messaging groups. We further refined the dataset through a human evaluation phase, addressing linguistic inconsistencies and ensuring accuracy in code-switching, phonetic representations, and synonym variability. The resulting dataset captures Roman Urdu's diverse linguistic features and serves as a critical resource for machine translation, sentiment analysis, and multilingual education.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。