ChartInsighter:一种借助基准数据集缓解时序图表摘要生成中幻觉问题的方法
ChartInsighter: An Approach for Mitigating Hallucination in Time-series Chart Summary Generation with A Benchmark Dataset
AI总结:
针对时序图表摘要生成的幻觉问题,提出多智能体协作结合外部分析模块与自一致性检验的ChartInsighter方法,构建逐句标注幻觉的基准,验证其优于SOTA模型、幻觉率最低。
摘要:
有效的图表摘要可大幅减少决策者解读图表的时间与精力,实现数据洞察的精准高效传递。既往研究在生成准确且语义丰富的时序数据图表摘要方面面临挑战。本文明确了时序图表摘要生成中的摘要要素与常见幻觉类型,将其作为自动生成的指导准则。我们提出ChartInsighter,可自动生成时序数据的图表摘要,有效减少摘要生成中的幻觉。具体而言,我们指派多个智能体生成初始图表摘要并迭代协作,过程中它们调用外部数据分析模块提取洞察,并将其整合成连贯的摘要。此外,我们实现了一种自一致性检验方法来验证并修正摘要。我们构建了一个高质量的图表与摘要基准,逐句标注了幻觉类型,为评估幻觉缓解效果提供支持。基于该基准的评估显示,我们的方法优于现有最优模型,且摘要幻觉率最低,可有效减少各类幻觉并提升摘要质量。该基准可在https://github.com/wangfen01/ChartInsighter获取。
英文摘要:
Effective chart summary can significantly reduce the time and effort decision makers spend interpreting charts, enabling precise and efficient communication of data insights. Previous studies have faced challenges in generating accurate and semantically rich summaries of time-series data charts. In this paper, we identify summary elements and common hallucination types in the generation of time-series chart summaries, which serve as our guidelines for automatic generation. We introduce ChartInsighter, which automatically generates chart summaries of time-series data, effectively reducing hallucinations in chart summary generation. Specifically, we assign multiple agents to generate the initial chart summary and collaborate iteratively, during which they invoke external data analysis modules to extract insights and compile them into a coherent summary. Additionally, we implement a self-consistency test method to validate and correct our summary. We create a high-quality benchmark of charts and summaries, with hallucination types annotated on a sentence-by-sentence basis, facilitating the evaluation of the effectiveness of reducing hallucinations. Our evaluations using our benchmark show that our method surpasses state-of-the-art models, and that our summary hallucination rate is the lowest, which effectively reduces various hallucinations and improves summary quality. The benchmark is available at https://github.com/wangfen01/ChartInsighter.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。