Large language models (LLMs) are known for their exceptional performance across a range of natural language processing tasks, but their deployment comes at a high computational and financial cost. On the other hand, smaller language models (SLMs), which can be deployed on lower-cost edge devices, struggle to match the performance of their larger counterparts. This paper presents a novel hybrid inference approach that leverages the strengths of both model types while minimizing reliance on costly cloud-based LLMs. Unlike existing methods that route entire queries to either an SLM or a cloud LLM, our approach introduces a reward-based mechanism to dynamically determine the involvement of the cloud LLM during token generation. Specifically, each token predicted by the SLM is evaluated against a reward score, and only when this score falls below a certain threshold is the cloud LLM consulted for assistance in the next token prediction. This method not only reduces the traffic to the cloud LLM, thereby lowering costs, but also allows for flexible control over response quality depending on the reward score threshold. Experimental results demonstrate that our approach significantly reduces cloud LLM usage with minimal impact on overall response quality, offering a cost-effective solution for deploying high-performance language models
Selective Preference Optimization via Token-Level Reward Function Estimation
National Centre for Text Mining, The University of Manchester（曼彻斯特大学文本挖掘中心）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Center for Language and Information Research, Wuhan University（武汉大学语言与信息研究中心）；The Fin AI（Fin AI公司）；Baidu Inc.（百度公司）；Archimedes Research（阿基米德研究）
From Coarse to Fine: Benchmarking and Reward Modeling for Writing-Centric Generation Tasks
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University（数据科学上海重点实验室，计算机科学与人工智能学院，复旦大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Sub-SA: Strengthen In-context Learning via Submodular Selective Annotation
School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Automation, Southeast University（东南大学自动化学院）；State Key Laboratory of ASIC and System, Fudan University（复旦大学专用集成电路与系统国家重点实验室）
Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
School of Data Science, Fudan University（复旦大学数据科学学院）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University（华东师范大学上海智能教育研究院）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与技术学院）；Ant Group（蚂蚁集团）
Aligning Language Models with Selective Prediction
Department of Computer Science and Engineering（计算机科学与工程系）；Bioinformatics and Computational Biology Program（生物信息学与计算生物学项目）；Division of Biostatistics and Health Data Science（生物统计学与健康数据科学部）；University of Minnesota Twin Cities（明尼苏达大学双城分校）
Posterior-GRPO: Rewarding Reasoning Processes in Code Generation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）
