基于用户偏好的动态LLM路由与选择：平衡性能、成本与伦理
Dynamic LLM Routing and Selection based on User Preferences: Balancing Performance, Cost, and Ethics
AI总结：
针对不同任务对LLM需求差异，提出OptiRoute动态路由引擎，基于用户偏好平衡性能、成本与伦理，采用kNN与分层过滤实现高效模型选择。
摘要：
随着GPT4、BART和LLaMA等大型语言模型（LLM）的广泛部署，对一种能够智能选择最适合特定任务的模型，同时平衡成本、延迟、准确性和伦理考量的系统的需求日益重要。认识到并非所有任务都需要超过1000亿参数的模型，我们引入了OptiRoute，一种先进的模型路由引擎，旨在根据详细的用户定义需求动态选择并将任务路由到最优LLM。OptiRoute捕获功能标准（如准确性、速度、成本）和非功能标准（如有用性、无害性、诚实性），利用轻量级任务分析和复杂度估计，高效地将任务与来自多样化LLM集合中的最合适模型匹配。通过采用结合k近邻（kNN）搜索和分层过滤的混合方法，OptiRoute在优化用户优先级的同时最小化计算开销。这使得它非常适合云基础ML平台、个性化AI服务和受监管行业中的实时应用。
英文摘要：
With the widespread deployment of large language models (LLMs) such as GPT4, BART, and LLaMA, the need for a system that can intelligently select the most suitable model for specific tasks while balancing cost, latency, accuracy, and ethical considerations has become increasingly important. Recognizing that not all tasks necessitate models with over 100 billion parameters, we introduce OptiRoute, an advanced model routing engine designed to dynamically select and route tasks to the optimal LLM based on detailed user-defined requirements. OptiRoute captures both functional (e.g., accuracy, speed, cost) and non-functional (e.g., helpfulness, harmlessness, honesty) criteria, leveraging lightweight task analysis and complexity estimation to efficiently match tasks with the best-fit models from a diverse array of LLMs. By employing a hybrid approach combining k-nearest neighbors (kNN) search and hierarchical filtering, OptiRoute optimizes for user priorities while minimizing computational overhead. This makes it ideal for real-time applications in cloud-based ML platforms, personalized AI services, and regulated industries.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
