arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.15463cs.CLcs.AI
收藏

DataFrame QA:一种无需数据暴露的通用DataFrame问答大语言模型框架

DataFrame QA: A Universal LLM Framework on DataFrame Question Answering Without Data Exposure

New Jersey Institute of Technology(新泽西理工学院)

Junyi Ye, Mengnan Du, Guiling Wang

AI总结:

本研究提出DataFrame QA新型任务及对应框架,仅用数据列名让LLM生成安全Pandas查询实现数据分析,零样本部署下GPT-4在两个数据集上取得86%、97%的pass@1准确率,兼顾隐私与效率。

摘要:

本文提出了DataFrame问答(QA)这一新型任务,即利用大语言模型(LLM)生成Pandas查询语句,以在dataframe上执行信息检索与数据分析,重点强调安全且不泄露的数据处理方式。我们的方法仅依赖dataframe的列名,既保障了数据隐私,又大幅缩减了提示词中的上下文窗口,简化了信息处理流程,解决了基于LLM的数据分析中的核心挑战。我们提出的DataFrame QA是一套综合框架,涵盖安全的Pandas查询生成与代码执行功能。研究采用pass@1指标,在知名的WikiSQL数据集以及我们新构建的、面向复杂数据分析查询的“UCI-DataFrameQA”数据集上,对多款LLM(尤其是GPT-4)进行了评估。结果显示,GPT-4在WikiSQL上的pass@1准确率达86%,在UCI-DataFrameQA上达97%,印证了其在安全检索与聚合dataframe数值、执行复杂数据分析方面的能力。该方法可零样本部署,无需预训练或调整,适配性强且安全性高,适用于各类应用场景。

英文摘要:

This paper introduces DataFrame question answering (QA), a novel task that utilizes large language models (LLMs) to generate Pandas queries for information retrieval and data analysis on dataframes, emphasizing safe and non-revealing data handling. Our method, which solely relies on dataframe column names, not only ensures data privacy but also significantly reduces the context window in the prompt, streamlining information processing and addressing major challenges in LLM-based data analysis. We propose DataFrame QA as a comprehensive framework that includes safe Pandas query generation and code execution. Various LLMs, notably GPT-4, are evaluated using the pass@1 metric on the renowned WikiSQL and our newly developed 'UCI-DataFrameQA', tailored for complex data analysis queries. Our findings indicate that GPT-4 achieves pass@1 rates of 86% on WikiSQL and 97% on UCI-DataFrameQA, underscoring its capability in securely retrieving and aggregating dataframe values and conducting sophisticated data analyses. This approach, deployable in a zero-shot manner without prior training or adjustments, proves to be highly adaptable and secure for diverse applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑