CreditAudit: 2$^\text{nd}$ Dimension for LLM Evaluation and Selection
Northwestern Polytechnical University Institute of Artificial Intelligence (TeleAI), China Telecom(西北工业大学人工智能研究所(TeleAI),中国电信);Dalian Maritime University Institute of Artificial Intelligence (TeleAI), China Telecom(大连海事大学人工智能研究所(TeleAI),中国电信);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究所(TeleAI),中国电信);Guangxi Normal University Institute of Artificial Intelligence (TeleAI), China Telecom(广西师范大学人工智能研究所(TeleAI),中国电信)
Yiliang Song, Hongjun An, Jiangong Xiao, Haofei Zhao, Jiawei Shao, Xuelong Li
Leaderboard scores on public benchmarks have been steadily rising and converging, with many frontier language models now separated by only marginal differences. However, these scores often fail to match users' day to day experience, because system prompts, output protocols, and interaction modes evolve under routine iteration, and in agentic multi step pipelines small protocol shifts can trigger disproportionate failures, leaving practitioners uncertain about which model to deploy. We propose CreditAudit, a deployment oriented credit audit framework that evaluates models under a family of semantically aligned and non adversarial system prompt templates across multiple benchmarks, reporting mean ability as average performance across scenarios and scenario induced fluctuation sigma as a stability risk signal, and further mapping volatility into interpretable credit grades from AAA to BBB via cross model quantiles with diagnostics that mitigate template difficulty drift. Controlled experiments on GPQA, TruthfulQA, and MMLU Pro show that models with similar mean ability can exhibit substantially different fluctuation, and stability risk can overturn prioritization decisions in agentic or high failure cost regimes. By providing a 2D and grade based language for regime specific selection, CreditAudit supports tiered deployment and more disciplined allocation of testing and monitoring effort, enabling more objective and trustworthy model evaluation for real world use.
Answer Candidate Type Selection: Text-to-Text Language Model for Closed Book Question Answering Meets Knowledge Graphs
Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究院);Ural Federal University(乌拉尔联邦大学);Artificial Intelligence Research Institute(人工智能研究院);ISP RAS Research Center for Trusted Artificial Intelligence(俄罗斯科学院信息系统研究所可信人工智能研究中心)
Use Random Selection for Now: Investigation of Few-Shot Selection Strategies in LLM-based Text Augmentation for Classification
Faculty of Information Technology, Brno University of Technology(布尔诺理工大学信息技术学院);Kempelen Institute of Intelligent Technologies(肯佩伦智能技术研究所);University of Pittsburgh(匹兹堡大学)
Prompt Selection Matters: Enhancing Text Annotations for Social Sciences with Large Language Models
Université Paris 1 Panthéon-Sorbonne(巴黎第一大学先贤祠-索邦大学);Université Paris-Saclay(巴黎萨克雷大学);Institut Jean Nicod(让·尼科德研究所);Ecole Normale Supérieure(巴黎高等师范学院);PSL-EHESS-CNRS(巴黎文理研究大学-法国高等社会科学研究院-法国国家科学研究中心)
WisPerMed at "Discharge Me!": Advancing Text Generation in Healthcare with Large Language Models, Dynamic Expert Selection, and Priming Techniques on MIMIC-IV
University of Applied Sciences and Arts Dortmund(多特蒙德应用科学与艺术大学);University Hospital Essen(埃森大学医院);Bielefeld University of Applied Sciences and Arts(比勒费尔德应用科学与艺术大学)
CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference
King Abdullah University of Science;Centre for Artificial Intelligence;Robotics, Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences, Hong Kong, China
DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model
Xiamen University(厦门大学);Shanghai Jiao Tong University(上海交通大学);OpenGVLab, Shanghai AI Laboratory(上海人工智能实验室OpenGVLab);The University of Hong Kong(香港大学);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院)
UniICL: An Efficient Unified Framework Unifying Compression, Selection, and Generation
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Hong Kong Polytechnic University(香港理工大学);Stepfun
Combining Textual and Structural Information for Premise Selection in Lean
Faculty of Mathematics and Physics, University of Ljubljana(卢布尔雅纳大学数学与物理学院);University of Ljubljana(卢布尔雅纳大学);Department of Knowledge Technologies, Jožef Stefan Institute(知识技术系,乔泽夫·斯塔芬研究所)
Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection
University of Michigan(密歇根大学);University of New Mexico(新墨西哥大学);Emory University(埃默里大学);University of Washington(华盛顿大学);Allen Institute for AI(艾伦人工智能研究所)
Leveraging Estimated Transferability Over Human Intuition for Model Selection in Text Ranking
School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院);Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系)
Text Guide: Improving the quality of long text classification by a text selection method based on feature importance
University of Central Florida(中佛罗里达大学);Massachusetts Institute of Technology(麻省理工学院);University of Warsaw(华沙大学);Taif University(塔伊夫大学);University of Louisville(路易斯维尔大学)
T-TExTS (Teaching Text Expansion for Teacher Scaffolding): Enhancing Text Selection in High School Literature through Knowledge Graph-Based Recommendation
Department of Computer Science(计算机科学系);Department of Curriculum and Instruction(课程与教学系);Kansas State University(堪萨斯州立大学)
ADLM -- stega: A Universal Adaptive Token Selection Algorithm for Improving Steganographic Text Quality via Information Entropy
Faculty of Computer Science and Information Technology, Universiti Putra Malaysia(马来西亚博特拉大学计算机科学与信息技术学院);Research Institute of Social Development, Southwestern University of Finance and Economics, China(西南财经大学社会发展研究院)