arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.11734cs.PLcs.AIcs.SE
收藏

使用大型语言模型解决以数据为中心的任务

Solving Data-centric Tasks using Large Language Models

Microsoft(微软公司)
Shraddha Barke, Christian Poelitz, Carina Suzana Negreanu, Benjamin Zorn, José Cambronero, Andrew D. Gordon, Vu Le, Elnaz Nouri, Nadia Polikarpova, Advait Sarka… 展开作者

Shraddha Barke, Christian Poelitz, Carina Suzana Negreanu, Benjamin Zorn, José Cambronero, Andrew D. Gordon, Vu Le, Elnaz Nouri, Nadia Polikarpova, Advait Sarkar, Brian Slininger, Neil Toronto, Jack Williams

AI总结:

本文针对LLM解决表格数据操作任务时提示中数据选择的问题,创建了真实世界NL到代码任务数据集,并提出先聚类后选择的提示技术,实验证明其优于随机选择基线。

摘要:

大型语言模型(LLMs)正在迅速取代像StackOverflow这样的帮助论坛,并且对非专业程序员和终端用户特别有帮助。这些用户通常对以数据为中心的任务感兴趣,例如电子表格操作和数据整理,如果仅使用自然语言描述来传达意图而不包含数据,这些任务很难解决。但是我们如何决定在提示中包含多少数据以及包含哪些数据呢?本文在回答这个问题方面做出了两项贡献。首先,我们创建了一个真实世界NL到代码任务的数据集,这些任务操作表格数据,从StackOverflow帖子中挖掘。其次,我们引入了一种先聚类后选择的提示技术,该技术将输入数据中最具代表性的行添加到LLM提示中。我们的实验表明,LLM性能确实对提示中传递的数据量敏感,并且对于输入表中存在大量语法变化的任务,我们的先聚类后选择技术优于随机选择基线。

英文摘要:

Large language models (LLMs) are rapidly replacing help forums like StackOverflow, and are especially helpful for non-professional programmers and end users. These users are often interested in data-centric tasks, such as spreadsheet manipulation and data wrangling, which are hard to solve if the intent is only communicated using a natural-language description, without including the data. But how do we decide how much data and which data to include in the prompt? This paper makes two contributions towards answering this question. First, we create a dataset of real-world NL-to-code tasks manipulating tabular data, mined from StackOverflow posts. Second, we introduce a cluster-then-select prompting technique, which adds the most representative rows from the input data to the LLM prompt. Our experiments show that LLM performance is indeed sensitive to the amount of data passed in the prompt, and that for tasks with a lot of syntactic variation in the input table, our cluster-then-select technique outperforms a random selection baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑