arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.09281cs.AI
收藏

STaR:用于大推理模型中去学习的敏感轨迹调节

STaR: Sensitive Trajectory Regulation for Unlearning in Large Reasoning Models

Jingjing Zhou, Gaoxiang Cong, Li Su, Liang Li

AI总结:

STaR通过敏感轨迹调节方法,实现了大推理模型中高效且稳定的隐私保护去学习,减少隐私泄露风险。

摘要:

大型推理模型(LRMs)已推动了自动化多步骤推理的发展,但它们生成复杂链式思维(CoT)轨迹的能力引入了严重的隐私风险,因为敏感信息可能深入嵌入在推理过程中。现有的大型语言模型(LLMs)去学习方法通常只关注修改最终答案,对于LRMs来说是不够的,因为它们无法从中间步骤中删除敏感内容,导致持续的隐私泄露和安全下降。为了解决这些挑战,我们提出了敏感轨迹调节(STaR),一种参数无关、推理时间去学习框架,能够在推理过程中实现稳健的隐私保护。具体来说,我们首先通过语义感知检测识别敏感内容。然后,通过安全提示前缀注入全局安全约束。接下来,我们执行轨迹感知抑制,以动态阻止整个推理链中的敏感内容。最后,我们应用令牌级自适应过滤,以防止生成过程中出现精确和改写后的敏感令牌。此外,为了克服现有评估协议的不足,我们引入了两个指标:多解码一致性评估(MCS),用于衡量去学习在不同解码策略下的一致性,以及多粒度成员推断攻击(MIA)评估,用于量化在答案和推理链层面的隐私保护程度。在R-TOFU基准测试中,STaR实现了全面且稳定的去学习,具有最小的实用性损失,为LRMs中的隐私保护推理设定了新标准。

英文摘要:

Large Reasoning Models (LRMs) have advanced automated multi-step reasoning, but their ability to generate complex Chain-of-Thought (CoT) trajectories introduces severe privacy risks, as sensitive information may be deeply embedded throughout the reasoning process. Existing Large Language Models (LLMs) unlearning approaches that typically focus on modifying only final answers are insufficient for LRMs, as they fail to remove sensitive content from intermediate steps, leading to persistent privacy leakage and degraded security. To address these challenges, we propose Sensitive Trajectory Regulation (STaR), a parameter-free, inference-time unlearning framework that achieves robust privacy protection throughout the reasoning process. Specifically, we first identify sensitive content via semantic-aware detection. Then, we inject global safety constraints through secure prompt prefix. Next, we perform trajectory-aware suppression to dynamically block sensitive content across the entire reasoning chain. Finally, we apply token-level adaptive filtering to prevent both exact and paraphrased sensitive tokens during generation. Furthermore, to overcome the inadequacies of existing evaluation protocols, we introduce two metrics: Multi-Decoding Consistency Assessment (MCS), which measures the consistency of unlearning across diverse decoding strategies, and Multi-Granularity Membership Inference Attack (MIA) Evaluation, which quantifies privacy protection at both answer and reasoning-chain levels. Experiments on the R-TOFU benchmark demonstrate that STaR achieves comprehensive and stable unlearning with minimal utility loss, setting a new standard for privacy-preserving reasoning in LRMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑