LLM-dCache:通过GPT驱动的本地化数据缓存改进工具增强型LLM
LLM-dCache: Improving Tool-Augmented LLMs with GPT-Driven Localized Data Caching
AI总结:
本文提出LLM-dCache,将缓存操作作为可调用API暴露给工具增强型智能体,赋予LLMs通过提示自主管理缓存决策的能力,在工业级平台上将Copilot时间平均提升1.24倍。
摘要:
随着大型语言模型(LLMs)扩展其能力以管理数千次API调用,它们面临着跨庞大数据集的复杂数据操作,并给底层系统带来显著开销。在本工作中,我们引入LLM-dCache,通过将缓存操作作为暴露给工具增强型智能体的可调用API函数来优化数据访问。我们赋予LLMs通过提示管理缓存决策的自主权,并无缝集成到现有的函数调用机制中。在一个涵盖数百个GPT端点和数TB图像的工业级大规模并行平台上进行测试,我们的方法在各种LLMs和提示技术中将Copilot时间平均提升了1.24倍。
英文摘要:
As Large Language Models (LLMs) broaden their capabilities to manage thousands of API calls, they are confronted with complex data operations across vast datasets with significant overhead to the underlying system. In this work, we introduce LLM-dCache to optimize data accesses by treating cache operations as callable API functions exposed to the tool-augmented agent. We grant LLMs the autonomy to manage cache decisions via prompting, seamlessly integrating with existing function-calling mechanisms. Tested on an industry-scale massively parallel platform that spans hundreds of GPT endpoints and terabytes of imagery, our method improves Copilot times by an average of 1.24x across various LLMs and prompting techniques.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。