arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.11847cs.CL
收藏

Prompto:一个用于异步查询LLM端点的开源库

Prompto: An open source library for asynchronous querying of LLM endpoints

The Alan Turing Institute(阿兰·图灵研究所);University of Sheffield(谢菲尔德大学)

Ryan Sze-Yin Chan, Federico Nanni, Angus R. Williams, Edwin Brown, Liam Burke-Moore, Ed Chapman, Kate Onslow, Tvesha Sippy, Jonathan Bright, Evelina Gabasova

AI总结:

针对多LLM比较研究效率低的问题,提出开源Python库prompto,通过异步查询实现并发交互,提升实验效率与可重复性。

摘要:

近年来,大型语言模型(LLM)的普及为研究开辟了令人兴奋的途径。然而,高效地与这些模型交互却是一个重大障碍,因为LLM通常驻留在专有或自托管的API端点上,每个端点都需要自定义代码进行交互。因此,在不同模型之间进行比较研究可能非常耗时,并且需要大量的工程工作,从而阻碍了研究的效率和可重复性。为了解决这些挑战,我们提出了prompto,一个开源Python库,它支持对LLM端点进行异步查询,使研究人员能够同时与多个LLM交互,同时最大化效率并利用各自的速率限制。我们的库使研究人员和开发者能够更有效地与LLM交互,并允许更快的实验、数据生成和评估。prompto附带一个介绍视频(https://youtu.be/lWN9hXBOLyQ),在MIT许可证下发布,并可通过GitHub(https://github.com/alan-turing-institute/prompto)获取。

英文摘要:

Recent surge in Large Language Model (LLM) availability has opened exciting avenues for research. However, efficiently interacting with these models presents a significant hurdle since LLMs often reside on proprietary or self-hosted API endpoints, each requiring custom code for interaction. Conducting comparative studies between different models can therefore be time-consuming and necessitate significant engineering effort, hindering research efficiency and reproducibility. To address these challenges, we present prompto, an open source Python library which facilitates asynchronous querying of LLM endpoints enabling researchers to interact with multiple LLMs concurrently, while maximising efficiency and utilising individual rate limits. Our library empowers researchers and developers to interact with LLMs more effectively and allowing faster experimentation, data generation and evaluation. prompto is released with an introductory video (https://youtu.be/lWN9hXBOLyQ) under MIT License and is available via GitHub (https://github.com/alan-turing-institute/prompto).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑