arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.07652cs.AIcs.HC
收藏

通过LLM生成的声明性规范桥接自然语言与交互式假设分析接口

Bridging Natural Language and Interactive What-If Interfaces via LLM-Generated Declarative Specifications

University of Maryland, College Park(马里兰大学帕克分校);Adobe Research(Adobe研究院);Microsoft Research(微软研究院);AWS AI Labs(AWS人工智能实验室);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)

Sneha Gathani, Sirui Zeng, Diya Patel, Ryan Rossi, Dan Marshall, Cagatay Demiralp, Steven Drucker, Zhicheng Liu

AI总结:

本文提出一种两阶段工作流,通过Praxa规范语言将自然语言假设分析问题转化为交互式可视化界面,提升假设分析工具的交互性和准确性。

摘要:

假设分析(WIA)是一种迭代、多步骤的过程,用户通过调整参数、应用约束和限定数据来探索和比较假设场景。当前工具在支持有效的交互式WIA方面存在不足:电子表格和BI工具需要耗时且繁琐的设置,而基于LLM的聊天机器人接口在语义上脆弱,频繁误解意图并产生不一致的结果。为了解决这些限制,我们提出一个两阶段工作流,通过中间表示将自然语言(NL)WIA问题转换为交互式可视化界面,由Praxa规范语言(PSL)驱动:首先,LLM从NL问题生成PSL规范,捕捉分析意图和逻辑,使规范能够验证和修复错误;其次,规范被编译为具有参数控制和链接可视化效果的交互式界面。我们通过405个WIA问题(涵盖11种WIA类型、5个数据集和3种最先进的LLM)对工作流进行基准测试。结果表明,跨模型,52.42%的规范在无干预的情况下生成正确。我们分析了失败案例并推导出涵盖非功能错误(规范无法编译)和功能错误(规范编译但误解意图)的错误分类。基于此分类,我们使用少量示例提示对失败案例进行针对性修复,并将成功率提高到80.42%。最后,我们展示了未检测到的功能错误如何通过编译传播到合理但误导性的界面,证明中间规范对于可靠地桥接自然语言和交互式WIA接口在LLM驱动的WIA系统中至关重要。

英文摘要:

What-if analysis (WIA) lets users explore hypothetical scenarios by adjusting parameters, applying constraints, and scoping data through interactive interfaces. Current tools fall short: spreadsheet and BI tools require laborious setup, while LLM-generated interfaces frequently misinterpret analytical intent. It remains unclear whether an explicit intermediate representation is needed, or whether LLMs can generate WIA interfaces directly. We build a benchmark of 405 WIA questions across 11 types and 5 datasets with human-authored ground-truth intent, and find that direct one-shot generation across three LLMs fails to produce a working interface in 17-45% of cases and, when interfaces render, often misrepresents the question's objective (up to 77.5%) and constraints (up to 58.2%). We therefore present a two-stage workflow that translates NL questions into Praxa Specification Language (PSL) specifications, then compiles them into interfaces. Because errors localize to named properties, taxonomy-guided repair raises correctness from 52.42% to 80.42%, unlike errors in directly generated code.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑