arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.14062cs.HCcs.CY
收藏

对电子表格场景下生成式AI与大语言模型信任度的理解与评估

Understanding and Evaluating Trust in Generative AI and Large Language Models for Spreadsheets

Simon Thorne

AI总结:

针对生成式AI与LLMs生成电子表格公式的可信度存疑问题,提出涵盖透明度、可靠性维度的可信性评估框架,分析影响因素并探讨技术不信任的案例与后果。

摘要:

生成式AI与大语言模型(LLMs)在自动化生成电子表格公式方面颇具潜力。但受幻觉、偏差以及用户技能水平参差不齐的影响,不能假定生成式AI的输出准确可信。为应对这些挑战,本文提出一套基于公式透明度与可靠性评估的可信性框架。其中,公式透明度通过可解释性(理解公式的推理逻辑)与可见性(检视底层算法)两个维度展开探究;生成公式的可靠性则从可信度(一致性与准确率)与伦理考量(偏差与公平性)两方面进行评估。本文还分析了幻觉、训练数据偏差、提示词构建不当等影响上述指标的驱动因素,最后探讨了技术不信任的相关案例及其后果。

英文摘要:

Generative AI and Large Language Models (LLMs) hold promise for automating spreadsheet formula creation. However, due to hallucinations, bias and variable user skill, outputs obtained from generative AI cannot be assumed to be accurate or trustworthy. To address these challenges, a trustworthiness framework is proposed based on evaluating the transparency and dependability of the formula. The transparency of the formula is explored through explainability (understanding the formula's reasoning) and visibility (inspecting the underlying algorithms). The dependability of the generated formula is evaluated in terms of reliability (consistency and accuracy) and ethical considerations (bias and fairness). The paper also examines the drivers to these metrics in the form of hallucinations, training data bias and poorly constructed prompts. Finally, examples of mistrust in technology are considered and the consequences explored.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑