HanjaBridge：通过汉字增强的预训练解决韩语LLM中的语义歧义
HanjaBridge: Resolving Semantic Ambiguity in Korean LLMs via Hanja-Augmented Pre-Training
AI总结：
提出HanjaBridge技术，通过在持续预训练中注入汉字候选以解决韩语同音词语义歧义，结合知识蒸馏防止遗忘，显著提升韩语理解并实现跨语言正向迁移。
摘要：
大型语言模型（LLMs）在韩语等低资源语言上通常表现不佳，部分原因在于独特的语言挑战，例如在韩文字母中无法区分的同音韩语汉字词。为了解决这种语义歧义，我们提出了HanjaBridge，这是一种集成到持续预训练（CPT）框架中的新型含义注入技术。HanjaBridge没有将单词确定性地映射到单个汉字，而是向模型呈现给定同音异义词的所有可能的汉字候选，鼓励模型学习上下文消歧。该过程与token级知识蒸馏相结合，以防止灾难性遗忘。实验结果表明，HanjaBridge显著提升了韩语理解能力，在KoBALT基准上实现了21%的相对提升。值得注意的是，通过共享汉字强化韩语和中文之间的语义对齐，我们观察到了强烈的正向跨语言迁移。此外，即使在推理时省略汉字增强，这些增益依然存在，确保了实际效率且没有额外的运行时成本。
英文摘要：
Large language models (LLMs) often show poor performance in low-resource languages like Korean, partly due to unique linguistic challenges such as homophonous Sino-Korean words that are indistinguishable in Hangul script. To address this semantic ambiguity, we propose HanjaBridge, a novel meaning-injection technique integrated into a continual pre-training (CPT) framework. Instead of deterministically mapping a word to a single Hanja (Chinese character), HanjaBridge presents the model with all possible Hanja candidates for a given homograph, encouraging the model to learn contextual disambiguation. This process is paired with token-level knowledge distillation to prevent catastrophic forgetting. Experimental results show that HanjaBridge significantly improves Korean language understanding, achieving a 21\% relative improvement on the KoBALT benchmark. Notably, by reinforcing semantic alignment between Korean and Chinese through shared Hanja, we observe a strong positive cross-lingual transfer. Furthermore, these gains persist even when Hanja augmentation is omitted at inference time, ensuring practical efficiency with no additional run-time cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
