SEED：使用大型语言模型进行特定领域的数据治理
SEED: Domain-Specific Data Curation With Large Language Models
AI总结：
本文提出SEED，一种LLM作为编译器的方法，通过自动结合LLM查询、向量缓存、生成代码和小模型构建混合执行流水线，在保证少样本性能的同时显著降低LLM调用成本，解决特定领域数据治理难题。
摘要：
为分析准备数据的数据治理任务对于将数据转化为可操作的洞察至关重要。然而，由于不同领域应用的多样化需求，通用的现成工具通常是不够的。因此，数据科学家通常必须开发针对数据集和任务量身定制的特定领域解决方案，例如编写特定领域的代码或在足够数量的标注样本上训练机器学习模型。这个过程以困难和耗时著称。我们提出了SEED，一种LLM作为编译器（LLM-as-compiler）的方法，通过大型语言模型（LLMs）自动生成特定领域的数据治理解决方案。一旦用户描述了任务、输入数据和预期输出，SEED编译器就会生成一个混合流水线，将LLM查询与更具成本效益的替代方案相结合，例如基于向量的缓存、LLM生成的代码以及在LLM标注数据上训练的小模型。SEED具有一个优化器，可自动从这四个LLM辅助模块中进行选择，并形成最适合当前任务的混合执行流水线。为了验证这种全新的、革命性的方法，我们在跨越5个以上数据治理任务的9个数据集上进行了实验。与在每个数据记录上使用LLM的解决方案相比，SEED实现了最先进或可比的少样本性能，同时显著减少了LLM调用的数量。
英文摘要：
Data curation tasks that prepare data for analytics are critical for turning data into actionable insights. However, due to the diverse requirements of applications in different domains, generic off-the-shelf tools are typically insufficient. As a result, data scientists often have to develop domain-specific solutions tailored to both the dataset and the task, e.g. writing domain-specific code or training machine learning models on a sufficient number of annotated examples. This process is notoriously difficult and time-consuming. We present SEED, an LLM-as-compiler approach that automatically generates domain-specific data curation solutions via Large Language Models (LLMs). Once the user describes a task, input data, and expected output, the SEED compiler produces a hybrid pipeline that combines LLM querying with more cost-effective alternatives, such as vector-based caching, LLM-generated code, and small models trained on LLM-annotated data. SEED features an optimizer that automatically selects from the four LLM-assisted modules and forms a hybrid execution pipeline that best fits the task at hand. To validate this new, revolutionary approach, we conducted experiments on $9$ datasets spanning over $5$ data curation tasks. In comparison to solutions that use the LLM on every data record, SEED achieves state-of-the-art or comparable few-shot performance, while significantly reducing the number of LLM calls.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
