arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.14404cs.CL
收藏

超越语言的知识:弥合多语言机器遗忘评估中的差距

Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluation

GSCST, Seoul National University(首尔国立大学GSCST);AIIS, Seoul National University(首尔国立大学AIIS);Department of Artificial Intelligence, Chung-Ang University(Chung-Ang大学人工智能系);Korean Surgical Researcher Foundation, Republic of Korea(韩国外科研究员基金会)

Kyomin Hwang, Hyeonjin Kim, Sangyeon Cho, Nojun Kwak

AI总结:

本文提出两种新指标评估多语言信息分布,分析不同语言对间信息一致去除,为多语言机器遗忘评估提供新视角。

摘要:

尽管LLM在商业服务中日益普及,但它们可能泄露敏感的个人身份信息(PII)。对于训练于多语言语料库的LLM,多语言机器遗忘(MMU)旨在去除跨语言的信息。然而,先前的MMU评估未能捕捉到此类跨语言信息分布,主要局限于单语言评估协议的直接扩展。为此,我们提出了两个指标来评估跨语言的信息分布:知识分离度评分(KSS)和知识持续度评分(KPS)。KSS衡量多语言环境下的整体遗忘质量,而KPS更具体地旨在评估不同语言对间信息的一致去除。我们使用这些指标评估了各种遗忘方法,并进行了全面分析。通过我们的研究,我们提供了关于MMU独特现象的见解,并为MMU评估提供了新的视角。

英文摘要:

While LLMs are increasingly used in commercial services, they pose privacy risks such as leakage of sensitive personally identifiable information (PII). For LLMs trained on multilingual corpora, Multilingual Machine Unlearning (MMU) aims to remove information across multiple languages. However, prior MMU evaluations fail to capture such cross-linguistic distribution of information, being largely limited to direct extensions of per-language evaluation protocols. To this end, we propose two metrics to evaluate the information spread across languages: the Knowledge Separability Score (KSS) and the Knowledge Persistence Score (KPS). KSS measures the overall unlearning quality across multiple languages, while KPS more specifically aims to assess consistent removal of information among different language pairs. We evaluated various unlearning methods in the multilingual setting with these metrics and conducted comprehensive analyses. Through our investigation, we provide insights into unique phenomena exclusive to MMU and offer a new perspective on MMU evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑