arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.01540cs.HCcs.AIcs.CL
收藏

在课堂中学习提示以理解AI的局限:一项初步研究

Learning to Prompt in the Classroom to Understand AI Limits: A pilot study

Università degli Studi di Milano Bicocca(米兰比可卡大学);Universitat Pompeu Fabra(庞培法布拉大学);University of Regensburg(雷根斯堡大学);Politecnico di Milano(米兰理工大学);Università degli Studi di Milano(米兰大学);Istituto per le Tecnologie Didattiche (ITD-CNR)(意大利国家研究委员会教育技术研究所)
Emily Theophilou, Cansu Koyuturk, Mona Yavari, Sathya Bursic, Gregor Donabauer, Alessia Telari, Alessia Testa, Raffaele Boiano, Davinia Hernandez-Leo, Martin Ru… 展开作者

Emily Theophilou, Cansu Koyuturk, Mona Yavari, Sathya Bursic, Gregor Donabauer, Alessia Telari, Alessia Testa, Raffaele Boiano, Davinia Hernandez-Leo, Martin Ruskov, Davide Taibi, Alessandro Gabbiadini, Dimitri Ognibene

AI总结:

本研究通过高中课堂初步干预,教授学生提示策略,以提升对LLM局限的理解,结果显示交互质量提高、负面情绪减少。

摘要:

人工智能(AI)的进步在应对健康、气候等紧迫社会问题方面展现出巨大潜力。大型语言模型(LLM)及其衍生的聊天机器人(如ChatGPT)极大地提升了AI系统的自然语言处理能力,使其能够处理前所未有的非结构化数据量。然而,随之而来的兴奋情绪也引发了负面看法,即便AI方法在健康、遗传学等领域已展现出显著贡献。造成这种情绪的一个关键因素是误导性的认知,即认为LLM可以毫不费力地跨领域提供解决方案,而忽视了其幻觉和推理限制等局限性。承认AI的可错性对于应对对LLM可能产生的错误建议所持的教条式过度自信的影响至关重要。同时,这也能减少对AI的恐惧和其他负面态度。这需要全面的AI素养干预措施,以教育公众了解LLM的约束和有效使用技巧,即提示策略。为此,在一所高中对21名学生进行了一项初步教育干预。该干预包括介绍关于智能、AI和LLM的高层次概念,随后进行涉及ChatGPT的实践练习,以创建自然的教育对话并应用既定的提示策略。初步结果令人鼓舞,包括对该活动的高度评价、与LLM交互质量的提升、负面AI情绪的减少,以及对局限性(特别是不可靠性、对命令理解有限导致回应不令人满意、以及展示灵活性有限)的更好把握。我们的目标是探索AI接受因素,并完善这一方法以用于未来更受控的研究。

英文摘要:

Artificial intelligence's (AI) progress holds great promise in tackling pressing societal concerns such as health and climate. Large Language Models (LLM) and the derived chatbots, like ChatGPT, have highly improved the natural language processing capabilities of AI systems allowing them to process an unprecedented amount of unstructured data. However, the ensuing excitement has led to negative sentiments, even as AI methods demonstrate remarkable contributions (e.g. in health and genetics). A key factor contributing to this sentiment is the misleading perception that LLMs can effortlessly provide solutions across domains, ignoring their limitations such as hallucinations and reasoning constraints. Acknowledging AI fallibility is crucial to address the impact of dogmatic overconfidence in possibly erroneous suggestions generated by LLMs. At the same time, it can reduce fear and other negative attitudes toward AI. This necessitates comprehensive AI literacy interventions that educate the public about LLM constraints and effective usage techniques, i.e prompting strategies. With this aim, a pilot educational intervention was performed in a high school with 21 students. It involved presenting high-level concepts about intelligence, AI, and LLMs, followed by practical exercises involving ChatGPT in creating natural educational conversations and applying established prompting strategies. Encouraging preliminary results emerged, including high appreciation of the activity, improved interaction quality with the LLM, reduced negative AI sentiments, and a better grasp of limitations, specifically unreliability, limited understanding of commands leading to unsatisfactory responses, and limited presentation flexibility. Our aim is to explore AI acceptance factors and refine this approach for more controlled future studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑