arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01770econ.GNq-fin.EC
收藏

试验生成式人工智能:ChatGPT真的提高了每个人的生产率吗?

Experimenting with Generative AI: Does ChatGPT Really Increase Everyone's Productivity?

Voraprapa Nakavachara, Tanapong Potipiti, Thanee Chaiwat

AI总结:

本文通过让经济学学生用泰语写作和用Stata分析数据的实验发现,ChatGPT平均提升表现并节省时间,但部分学生及高能力学生未必受益,凸显数字与人类技能的互补作用。

摘要:

近年来,ChatGPT、Gemini和MidJourney等生成式AI技术取得了显著进展。近期文献记录了ChatGPT在其具备较强专业能力的领域对生产率的积极影响,这归因于大量训练数据集,例如英语以及Python/SQL编程。然而,关于ChatGPT在其能力仍可进一步提升的领域中的表现,文献仍然有限。本文旨在填补这一空白。我们开展了一项实验,要求经济学专业学生使用非英语语言(具体为泰语)完成写作分析任务,并使用较不常用的编程软件包(具体为Stata)完成数学与数据分析任务。研究结果表明,平均而言,参与者在使用ChatGPT时,在得分和完成任务所需时间方面表现更好。然而,详细考察显示,在使用ChatGPT时,34%的参与者在写作分析任务中没有改善,42%的参与者在数学与数据分析任务中没有改善。进一步调查表明,以计量经济学成绩作为代理指标的较高能力学生,在使用ChatGPT完成写作分析任务时表现反而更差。我们还发现有证据表明,具备更好数字技能的学生在使用ChatGPT时表现更好。本研究为生成式AI的影响提供了洞见。因此,利益相关者可以作出知情决策,以实施适当的政策框架或重新设计教育体系。研究还强调了人类技能在应对和补充技术局限方面的关键作用。

英文摘要:

Generative AI technologies such as ChatGPT, Gemini, and MidJourney have made remarkable progress in recent years. Recent literature has documented ChatGPT's positive impact on productivity in areas where it has strong expertise, attributable to extensive training datasets, such as the English language and Python/SQL programming. However, there is still limited literature regarding ChatGPT's performance in areas where its capabilities could still be further enhanced. This paper aims to fill this gap. We conducted an experiment in which economics students were asked to perform writing analysis tasks in a non-English language (specifically, Thai) and math & data analysis tasks using a less frequently used programming package (specifically, Stata). The findings suggest that, on average, participants performed better using ChatGPT in terms of scores and time taken to complete the tasks. However, a detailed examination reveals that 34% of participants saw no improvement in writing analysis tasks, and 42% did not improve in math & data analysis tasks when employing ChatGPT. Further investigation indicated that higher-ability students, as proxied by their econometrics grades, were the ones who performed worse in writing analysis tasks when using ChatGPT. We also found evidence that students with better digital skills performed better with ChatGPT. This research provides insights on the impact of generative AI. Thus, stakeholders can make informed decisions to implement appropriate policy frameworks or redesign educational systems. It also highlights the critical role of human skills in addressing and complementing the limitations of technology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑