arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.15052cs.AI
收藏

GlitchMiner:通过基于梯度的离散优化在大语言模型中挖掘故障令牌

GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization

Zihui Wu, Haichang Gao, Ping Wang, Shudong Zhang, Zhaoxiang Liu, Shiguo Lian

AI总结:

本文提出GlitchMiner框架,通过梯度引导局部搜索最大化预测熵,高效挖掘大语言模型中的故障令牌,在多个模型上优于现有方法。

摘要:

故障令牌(Glitch tokens)是指会触发大语言模型(LLMs)产生不可预测或异常行为的输入,对模型可靠性和安全性构成重大挑战。现有检测方法主要依赖启发式嵌入模式或内部表示中的统计异常,这限制了它们在不同模型架构之间的泛化能力,并可能遗漏偏离已观察模式的异常。我们提出GlitchMiner,一个以行为驱动的框架,旨在通过最大化预测熵来识别故障令牌。GlitchMiner利用梯度引导的局部搜索策略,高效探索离散令牌空间,而不依赖模型特定的启发式方法或大批量采样。在来自五个主要模型家族的十个LLMs上进行的大量实验表明,GlitchMiner在检测准确率和查询效率上持续优于现有方法,为有效的故障令牌发现提供了一种可泛化且可扩展的解决方案。代码可在[https://github.com/wooozihu/GlitchMiner]获取。

英文摘要:

Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner]

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑