思维划分：利用混合语言模型协同实现高效的端侧智能体
Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents
AI总结：
本文提出Division-of-Thoughts框架，通过任务分解、依赖调度和即插即用适配器协同本地小模型与云端大模型，在保持推理准确性的同时将推理时间和API成本分别降低66.12%和83.57%。
摘要：
网络内容的快速扩张使端侧AI助手成为帮助用户管理日益复杂的在线任务所不可或缺的工具。大语言模型中涌现的推理能力为下一代端侧AI智能体提供了一条有前景的路径。然而，在资源受限的本地设备上部署全规模大语言模型（LLMs）具有挑战性。在本文中，我们提出了Division-of-Thoughts（DoT），这是一种协作推理框架，利用本地部署的较小规模语言模型（SLMs）与基于云的大语言模型之间的协同作用。DoT利用任务分解器（Task Decomposer）激发语言模型固有的规划能力，将用户查询分解为更小的子任务，从而使混合语言模型能够充分发挥各自优势。此外，DoT采用任务调度器（Task Scheduler）分析子任务之间的两两依赖关系并构建依赖图，促进子任务的并行推理和关键步骤的识别。为了根据子任务难度分配适当的模型，DoT利用即插即用适配器（Plug-and-Play Adapter），这是一个附加在SLM上的额外任务头，不会改变SLM的参数。为了提升适配器的任务分配能力，我们提出了一种仅依赖任务执行反馈的自强化训练方法。在多个基准上的大量实验表明，我们的DoT在保持有竞争力的推理准确性的同时显著降低了LLM成本。具体而言，DoT将平均推理时间和API成本分别降低了66.12%和83.57%，同时实现了与最佳基线方法相当的推理准确性。
英文摘要：
The rapid expansion of web content has made on-device AI assistants indispensable for helping users manage the increasing complexity of online tasks. The emergent reasoning ability in large language models offer a promising path for next-generation on-device AI agents. However, deploying full-scale Large Language Models (LLMs) on resource-limited local devices is challenging. In this paper, we propose Division-of-Thoughts (DoT), a collaborative reasoning framework leveraging the synergy between locally deployed Smaller-scale Language Models (SLMs) and cloud-based LLMs. DoT leverages a Task Decomposer to elicit the inherent planning abilities in language models to decompose user queries into smaller sub-tasks, which allows hybrid language models to fully exploit their respective strengths. Besides, DoT employs a Task Scheduler to analyze the pair-wise dependency of sub-tasks and create a dependency graph, facilitating parallel reasoning of sub-tasks and the identification of key steps. To allocate the appropriate model based on the difficulty of sub-tasks, DoT leverages a Plug-and-Play Adapter, which is an additional task head attached to the SLM that does not alter the SLM's parameters. To boost adapter's task allocation capability, we propose a self-reinforced training method that relies solely on task execution feedback. Extensive experiments on various benchmarks demonstrate that our DoT significantly reduces LLM costs while maintaining competitive reasoning accuracy. Specifically, DoT reduces the average reasoning time and API costs by 66.12% and 83.57%, while achieving comparable reasoning accuracy with the best baseline methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
