The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or provide useful training signal. For instance, on LiveCodeBench, frontier models achieve over 99% Pass@1 on easy splits and exceed 90% Pass@1 on average across difficulty levels. Constructing new, challenging datasets typically requires substantial human effort, creating a bottleneck for progress. We introduce BenchEvolver, a solution-centric evolutionary framework that automatically transforms existing coding problems into harder variants. Rather than generating problems from scratch, BenchEvolver evolves reference solutions through structured transformations and derives corresponding statements and tests from the evolved solutions. This design grounds generation in executable semantics, enabling scalable construction of high-quality, diverse, and difficult tasks with verifiable correctness. Applying BenchEvolver to LiveCodeBench and SciCode, we obtain evolved tasks that are substantially harder while maintaining validity, reference correctness, and diversity. We further curate LiveCodeBench-Plus, a 91-problem benchmark combining evolved and difficult original LCB-v6 tasks, where frontier-model Pass@1 ranges from 27.5% to 62.6%, restoring clear discrimination among strong coding models. Importantly, evolved tasks remain challenging even for the model that generates them, enabling self-improvement. We further show that RL on evolved LCB tasks improves held-out coding performance: for gpt-oss-20b, seed+evolved training achieves +8.7 and +8.3 Pass@1 gains on LCB v6 Hard and LCB-Pro Easy, exceeding seed-only gains by 70.7% and 34.8%, respectively. Our results show that BenchEvolver can convert saturated benchmarks into frontier-level evaluation suites and reusable training signal.
Desert Camels and Oil Sheikhs: Arab-Centric Red Teaming of Frontier LLMs
MBZUAI(穆罕默德·本·扎耶德人工智能大学);University of Khartoum(喀土穆大学);Universität des Saarlandes(萨尔大学);University of British Columbia(不列颠哥伦比亚大学);Invertible AI(可逆人工智能公司);Vector Institute(向量研究所)
Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo(宁波数字孪生研究院,东方理工大学(宁波));Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算学系);Munich Center for Machine Learning, LMU Munich(慕尼黑大学机器学习慕尼黑中心)
AutoDataBench: A Data-centric Testbed for Accelerating Auto Research
The Hong Kong Polytechnic University(香港理工大学);IQuest Research;Shanghai Jiaotong University(上海交通大学);TU Darmstadt(达姆施塔特工业大学);The University of Manchester(曼彻斯特大学);University of Macau(澳门大学);Shanghai University of Finance and Economics(上海财经大学);Beihang University(北京航空航天大学)
Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models
Nanjing University(南京大学);Tsinghua University(清华大学);Hong Kong Polytechnic University(香港理工大学);Johns Hopkins University(约翰斯·霍普金斯大学);New York University(纽约大学);NVIDIA(英伟达)
Green Shielding: A User-Centric Approach Towards Trustworthy AI
University of California, Berkeley(加州大学伯克利分校);University of Melbourne(墨尔本大学);University of California, San Francisco(加州大学旧金山分校);University of Georgia(佐治亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms
Institute of Information Technology, University of Dhaka(达卡大学信息技术学院);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校);Bangladesh Air Force Shaheen College Kurmitola(孟加拉国空军库尔米托拉沙欣学院);Ciroos Inc.(Ciroos公司)
Tracing the Arrow of Time: Diagnosing Temporal Information Flow in Video-LLMs
The University of Osaka(大阪大学);Center for Information and Neural Networks(信息与神经网络中心);National Institute of Information and Communications Technology(信息与通信技术国家研究所);Kyoto University(京都大学);NII LLMC(日本信息处理学会LLMC)
A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Monash University(莫纳什大学);Fudan University(复旦大学);Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学);University College London(伦敦大学学院);UNC-Chapel Hill(北卡罗来纳大学教堂山分校);Stanford University(斯坦福大学);Virginia Tech(弗吉尼亚理工大学);Purdue University(普渡大学);China Pharmaceutical University(中国药科大学);Beijing Institute of Heart, Lung and Blood Vessel Diseases(北京市心肺血管疾病研究所);Chinese Academy of Sciences(中国科学院);Johns Hopkins University(约翰斯·霍普金斯大学);Fuzhou University(福州大学);University of New South Wales(新南威尔士大学);University College Dublin(都柏林大学学院);The University of Hong Kong(香港大学);The Hong Kong Polytechnic University(香港理工大学);The Hong Kong University of Science and Technology(香港科技大学);South China University of Technology(华南理工大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Caltech(加州理工学院);North University of China(中北大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);University of Cambridge(剑桥大学);Nanjing University(南京大学)
LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?
New York University(纽约大学);Princeton University(普林斯顿大学);University of California San Diego(加利福尼亚大学圣地亚哥分校);University of Washington(华盛顿大学);Canyon Crest Academy(峡谷克雷斯特学院);University of Waterloo(滑铁卢大学);McGill University(麦吉尔大学)