发表机构
Northwest A&F University; East China Normal University; Renmin University of China; PingCAP（西北农林科技大学; 华东师范大学; 中国人民大学; 平凯星辰）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
WeaveData是一个多模态数据分析系统，通过自我批评、自我进化计划及元数据知识图谱，提高LLM生成分析计划的准确性，并在两个公开数据集上验证了其有效性。
AI 中文摘要
多模态数据分析，即回答涉及关系表、文本和图像的问题，在数据管理领域引起了越来越多的关注。大型语言模型（LLMs）通过生成基于关系和语义运算符的分析计划，使得这种分析能够以自然语言进行。然而，LLM生成的计划容易出错：一个计划可能默默计算出与问题要求不符的内容，在执行过程中失败，或返回遗漏问题的结果。本文介绍了WeaveData，一个具有自我批评与自我进化LLM计划的多模态数据分析系统。首先，WeaveData为每个问题生成一个类型化的逻辑计划，并在执行前逐步对其进行批评，之后还会对照问题检查执行结果。其次，WeaveData会进化那些失败或遗漏问题的计划：它利用实际数据诊断失败原因，重用仍然有效的结果，并为后续问题积累规划经验。第三，WeaveData将规划基于所有模态的元数据知识图谱，与用户澄清模糊问题，并在交互式笔记本中为每个模型判断提供证据支持。我们在两个公开的多模态数据集上展示了WeaveData。
英文摘要
Multimodal data analysis, which answers questions over relational tables, text, and images, has attracted growing attention in the data management community. Large language models (LLMs) enable such analysis in natural language by generating analysis plans over relational and semantic operators. However, LLM-generated plans are error-prone: a plan may silently compute something other than what was asked, fail during execution, or return a result that misses the question. This paper presents WeaveData, a multimodal data analysis system with self-critiquing and self-evolving LLM plans. First, WeaveData generates a typed logical plan for each question and critiques it step by step before execution, and it checks the executed result against the question afterwards. Second, WeaveData evolves a plan that fails or misses the question: it diagnoses the failure with the actual data, reuses the results that remain valid, and accumulates planning experience for later questions. Third, WeaveData grounds planning in a metadata knowledge graph of all modalities, clarifies ambiguous questions with the user, and backs every model judgment with evidence in an interactive notebook. We demonstrate WeaveData on two public multimodal datasets.
Comments5 pages, 2 figures
