arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.04795cs.CLcs.AI
收藏

SemEval-2025任务4中AI的网络:被遗忘但未丢失:大语言模型中选择性遗忘的平衡之道

Cyber for AI at SemEval-2025 Task 4: Forgotten but Not Lost: The Balancing Act of Selective Unlearning in Large Language Models

TCS Research(塔塔咨询服务研究中心)

Dinesh Srivasthav P, Bala Mallikarjunarao Garlapati

AI总结:

本文针对大语言模型选择性遗忘问题,提出基于全局权重修改的方法以平衡遗忘效果、知识保留与模型实用性,并在SemEval 2025任务4中取得有前景的测试结果。

摘要:

当敏感或过时数据必须被选择性删除时,大语言模型(LLMs)在维护隐私、伦理和合规性方面面临重大挑战。从头重新训练这些模型在计算上是不可行的,因此需要高效的替代方案。作为SemEval 2025任务4的一部分,本工作聚焦于在LLMs中应用选择性遗忘来解决这一挑战。在本文中,我们展示了实验与发现,主要利用全局权重修改来在遗忘有效性、知识保留以及目标模型遗忘后的实用性之间实现平衡。我们还详细介绍了任务特定的评估机制、结果和挑战。我们的算法在测试集上针对7B和1B目标模型分别取得了0.409和0.389的综合得分,在可验证的LLM遗忘方面展示了有前景的结果。

英文摘要:

Large Language Models (LLMs) face significant challenges in maintaining privacy, ethics, and compliance, when sensitive or obsolete data must be selectively removed. Retraining these models from scratch is computationally infeasible, necessitating efficient alternatives. As part of the SemEval 2025 Task 4, this work focuses on the application of selective unlearning in LLMs to address this challenge. In this paper, we present our experiments and findings, primarily leveraging global weight modification to achieve an equilibrium between effectiveness of unlearning, knowledge retention, and target model's post-unlearning utility. We also detail the task-specific evaluation mechanism, results, and challenges. Our algorithms have achieved an aggregate score of 0.409 and 0.389 on the test set for 7B and 1B target models, respectively, demonstrating promising results in verifiable LLM unlearning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑