Pass@$k$ is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, and short-answer reasoning. It defines success if any of $k$ independently sampled solutions passes a verifier. This multi-sample inference metric has motivated inference-aware fine-tuning methods that directly optimize pass@$k$. However, prior work reports a trade-off: pass@$k$ improves while pass@1 degrades under such methods. This trade-off is practically important because pass@1 often remains a hard operational constraint due to latency and cost budgets, imperfect verifier coverage, and the need for a reliable single-shot fallback. We study the origin of this trade-off and provide a theoretical characterization of when pass@$k$ policy optimization can reduce pass@1 through gradient conflict induced by prompt interference. This interference arises because shared model parameters couple prompts, so improving performance on some prompts can adversely affect others. We show that pass@$k$ policy gradients can conflict with pass@1 gradients because pass@$k$ optimization implicitly reweights prompts toward low-success prompts; when these prompts are negatively interfering, their upweighting can rotate the pass@$k$ update direction away from the pass@1 direction. We illustrate our findings with large language model experiments on verifiable mathematical reasoning and coding tasks.
Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve?
Concordia University(康考迪亚大学);Mila-Quebec AI Institute(米拉-魁北克人工智能研究所);University of Tübingen(蒂宾根大学);Tübingen AI Center(蒂宾根人工智能中心);Cohere For AI(Cohere人工智能项目);Laval University(拉瓦尔大学)
Learning, Social Intelligence and the Turing Test - why an "out-of-the-box" Turing Machine will not pass the Turing Test
Centre for Policy Modelling, Manchester Metropolitan University(曼彻斯特城市大学政策建模中心);Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas, Universidad Nacional Autónoma de México(墨西哥国立自治大学应用数学与系统研究所)
Why Does RL Generalize Better Than SFT? A Data-Centric Perspective on VLM Post-Training
College of Computer Science, Sichuan University(四川大学计算机科学学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院)