arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.07441cs.CL
收藏

基于ChatGPT的情感分析在AI质量保障中的稳定性分析

Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurance

National Institute of Informatics(国立情报学研究所);National Institute of Advanced Industrial Science and Technology, Japan(日本产业技术综合研究所)

Tinghui Ouyang, AprilPyone MaungMaung, Koichi Konishi, Yoshiki Seo, Isao Echizen

AI总结:

本文针对基于ChatGPT的情感分析系统,通过基准数据集实验分析了其在运行和鲁棒性方面的稳定性问题,揭示了系统存在不确定性及面对小文本攻击时的稳定性缺陷。

摘要:

在大型AI模型时代,复杂的架构和庞大的参数给有效的AI质量管理(AIQM)带来了重大挑战,例如大语言模型(LLM)。本文重点研究一个基于LLM的特定AI产品的质量保障——一个基于ChatGPT的情感分析系统。该研究深入探讨了ChatGPT所基于的庞大AI模型在运行和鲁棒性方面的稳定性问题。使用情感分析基准数据集进行了实验分析。结果表明,所构建的基于ChatGPT的情感分析系统表现出不确定性,这归因于各种运行因素。研究证明,该系统在处理涉及鲁棒性的常规小文本攻击时也表现出稳定性问题。

英文摘要:

In the era of large AI models, the complex architecture and vast parameters present substantial challenges for effective AI quality management (AIQM), e.g. large language model (LLM). This paper focuses on investigating the quality assurance of a specific LLM-based AI product--a ChatGPT-based sentiment analysis system. The study delves into stability issues related to both the operation and robustness of the expansive AI model on which ChatGPT is based. Experimental analysis is conducted using benchmark datasets for sentiment analysis. The results reveal that the constructed ChatGPT-based sentiment analysis system exhibits uncertainty, which is attributed to various operational factors. It demonstrated that the system also exhibits stability issues in handling conventional small text attacks involving robustness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑