The growing capabilities of large language models (LLMs) have led to the saturation of many benchmarks and training datasets used to improve them. Motivated by this, we investigate whether questions solved with perfect empirical accuracy can nevertheless be used to improve downstream performance. To do so, we replace binary correctness with two sources of more fine-grained quality signals: (1) pairwise LLM self-judgments, in which the model evaluates the relative quality of its own solutions, and (2) token-level entropy, where token-level uncertainty is used as a proxy for solution quality. We incorporate these signals into several training algorithms and evaluate them on Qwen3-1.7B-Base. When training exclusively on a simple arithmetic task, quality-based signals improve performance by up to $18.6\%$ over the base model, substantially outperforming SFT. On GSM8K, however, gains are more modest and depend strongly on the quality signal. For instance, self-judgments show poor agreement with a stronger external judge and can even degrade performance below the base model. Overall, our results suggest that quality-based training can extract useful signal from saturated questions for base models, but that applying such signals to more complex tasks requires careful calibration and further study.
Beyond Correctness: Benchmarking Multi-dimensional Code Generation for Large Language Models
University of Chinese Academy of Sciences(中国科学院大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Chinese Information Processing Laboratory(中文信息处理实验室);State Key Laboratory of Computer Science(计算机科学国家重点实验室);Key Laboratory of System Software, Chinese Academy of Sciences(中国科学院系统软件重点实验室)
Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence
Meituan(美团);The University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Nanjing University(南京大学);Harbin Institute of Technology(哈尔滨工业大学);Australian Institute for Machine Learning, Adelaide University(阿德莱德大学澳大利亚机器学习研究所);Ludwig Maximilian University of Munich(慕尼黑大学);University of Science and Technology of China(中国科学技术大学);Queen Mary University of London(伦敦玛丽女王大学)
Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation
National University of Singapore(新加坡国立大学);Yunnan University(云南大学);State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院人工智能安全国家重点实验室,计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Measuring Reasoning Quality in LLMs: A Multi-Dimensional Behavioral Framework
Department of Computer Engineering, Tarsus University(塔鲁斯大学计算机工程系);School of Computing and Augmented Intelligence (SCAI), Arizona State University (ASU)(计算与增强智能学院(SCAI),亚利桑那州立大学(ASU));HumaConn AI Consulting(HumaConn AI咨询)