arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.13069cs.CLcs.IR
收藏

基于多数投票的本地大语言模型动态情感分析:影响餐厅评价因素的研究

Dynamic Sentiment Analysis with Local Large Language Models using Majority Voting: A Study on Factors Affecting Restaurant Evaluation

Meijo University(名城大学);RIKEN AIP(理化学研究所人工智能项目)

Junichiro Niimi

AI总结:

本研究将多数投票机制引入基于本地大语言模型的情感分析,通过餐厅评论的三项分析证明中等规模模型多次尝试的多数投票比大规模模型单次尝试更稳健,并探究各因素对整体评价的影响。

摘要:

在线平台上的用户生成内容(UGC)使营销研究人员能够了解消费者对产品和服务的偏好。随着大语言模型(LLMs)的发展,一些研究利用这些模型进行标注和情感分析。然而,模型准确性与超参数之间的关系尚未得到彻底研究。此外,现有文献很少考虑每次LLM试验结果的变异性和可重复性问题。由于实际人工标注使用多数投票来解决标注者之间的分歧,本研究将多数投票机制引入到使用本地LLMs的情感分析模型中。通过对餐厅评价在线评论的三项分析,我们证明使用中等规模模型进行多次尝试的多数投票比使用大规模模型进行单次尝试产生更稳健的结果。此外,我们进行了进一步分析,以调查每个方面对整体评价的影响。

英文摘要:

User-generated contents (UGCs) on online platforms allow marketing researchers to understand consumer preferences for products and services. With the advance of large language models (LLMs), some studies utilized the models for annotation and sentiment analysis. However, the relationship between the accuracy and the hyper-parameters of LLMs is yet to be thoroughly examined. In addition, the issues of variability and reproducibility of results from each trial of LLMs have rarely been considered in existing literature. Since actual human annotation uses majority voting to resolve disagreements among annotators, this study introduces a majority voting mechanism to a sentiment analysis model using local LLMs. By a series of three analyses of online reviews on restaurant evaluations, we demonstrate that majority voting with multiple attempts using a medium-sized model produces more robust results than using a large model with a single attempt. Furthermore, we conducted further analysis to investigate the effect of each aspect on the overall evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑