RSIV-Harness:一种用于开放域对话式文本转可视化的递归自我改进框架
RSIV-Harness: A Recursive Self-Improvement Harness Framework for Open-Domain Conversational Text-to-Visualization
AI总结:
提出RSIV-Harness框架,通过递归自我改进优化静态智能体框架,结合ODAC-Vis基准实验,验证其可提升文本转可视化任务的可执行性与图表结构等指标。
摘要:
用户经常在不知道哪个数据库或表包含相关数据的情况下请求图表,这类请求可能存在歧义,用户会提出后续问题以深化分析或迭代优化现有图表。因此,一个实用的助手必须在异构数据库中发现相关数据源,在需要时提出澄清问题,并跟踪不断变化的用户意图,同时保留之前对话轮次中建立的约束。然而,近期的文本转可视化智能体通常依赖于预定义的静态智能体框架,该框架在任务反复失败时仍保持不变。为解决这一局限,我们引入RSIV-Harness,该框架保持语言模型权重固定,通过智能体自身的失败痕迹对周围框架应用递归自我改进。为平衡灵活的自我改进与稳定性,我们提出四层框架,通过锚点检查和记忆检查评估提议的框架修订,对通过检查的修订进行优化,否则恢复经最佳验证的框架。我们还引入ODAC-Vis,这是一个包含640项任务的开放域、歧义性及对话式文本转可视化基准,共同评估开放域数据发现、歧义解决、多轮可视化优化及可执行图表生成。实验表明,与初始框架相比,RSIV-Harness的一轮改进提升了所有六项评估指标,其中可执行性(+28.02分)和图表结构(+34.01分)的提升尤为显著,证明其具备从任务解决经验中学习并将任务间的反复失败转化为显著性能提升的能力。
英文摘要:
Users often request charts without knowing which database or table holds the relevant data. Such requests may be ambiguous, and users may ask follow-up questions to deepen their analysis or iteratively refine existing charts. A useful assistant must therefore discover relevant data sources among heterogeneous databases, ask clarifying questions when needed, and track evolving user intent while preserving constraints established in earlier turns. Yet recent text-to-visualization agents typically rely on static agent harnesses that are predefined and remain unchanged despite recurring failures across tasks. To address this limitation, we introduce RSIV-Harness, which keeps the language model weights fixed and applies Recursive Self-Improvement to revise the surrounding harness from the agent's own failure traces. To balance flexible self-improvement with stability, we introduce a four-layer framework that evaluates proposed harness revisions through anchor and memory checks, refining those that pass and restoring the best validated harness otherwise. We further introduce ODAC-Vis, a benchmark with 640 tasks for Open-Domain, Ambiguous, and Conversational text-to-visualization that jointly evaluates open-domain data discovery, ambiguity resolution, multi-turn visualization refinement, and executable chart generation. Experiments show that a single round of RSIV-Harness improves all six evaluation metrics relative to the seed harness, with particularly large gains in executability (+28.02 points) and chart structure (+34.01 points), demonstrating the ability to learn from task-solving experience and turn recurring failures across tasks into substantial performance improvements.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。