Large language models (LLMs) often appear to excel on public benchmarks, but these high scores may mask an overreliance on dataset-specific surface cues rather than true language understanding. We introduce the Chameleon Benchmark Overfit Detector (C-BOD), a meta-evaluation framework that systematically distorts benchmark prompts via a parametric transformation and detects overfitting of LLMs. By rephrasing inputs while preserving their semantic content and labels, C-BOD exposes whether a model's performance is driven by memorized patterns. Evaluated on the MMLU benchmark using 26 leading LLMs, our method reveals an average performance degradation of 2.15% under modest perturbations, with 20 out of 26 models exhibiting statistically significant differences. Notably, models with higher baseline accuracy exhibit larger performance differences under perturbation, and larger LLMs tend to be more sensitive to rephrasings, indicating that both cases may overrely on fixed prompt patterns. In contrast, the Llama family and models with lower baseline accuracy show insignificant degradation, suggesting reduced dependency on superficial cues. Moreover, C-BOD's dataset- and model-agnostic design allows easy integration into training pipelines to promote more robust language understanding. Our findings challenge the community to look beyond leaderboard scores and prioritize resilience and generalization in LLM evaluation.
Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models
University of Chinese Academy of Sciences(中国科学院大学);Ricoh Software Research Center (Beijing) Co., Ltd.(理光软件研究中心(北京)有限公司);Chinese Information Processing Laboratory(中文信息处理实验室);State Key Laboratory of Computer Science(计算机科学国家重点实验室);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所)
BAPO: Base-Anchored Preference Optimization for Overcoming Forgetting in Large Language Models Personalization
Graduate School of AI, KAIST(韩国科学技术院人工智能研究生院);Samsung Advanced Institute of Technology(三星先进技术研究院);Department of Electrical and Computer Engineering, UBC(不列颠哥伦比亚大学电气与计算机工程系)
Surgical Knowledge Rewrite in Compact LLMs: An 'Unlearn-then-Learn' Strategy with ($IA^3$) for Localized Factual Modulation and Catastrophic Forgetting Mitigation
Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Sun Yat-Sen University(中山大学);Shenzhen MSU-BIT University(深圳北理莫斯科大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
Pattern Recognition Center, WeChat AI, Tencent Inc(腾讯公司微信AI模式识别中心);School of Engineering, Westlake University(西湖大学工程学院);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院先进技术研究所)
Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning
Hefei University of Technology(合肥工业大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);University of Amsterdam(阿姆斯特丹大学)