arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.11985cs.CLcs.CY
收藏

大型语言模型带来的气候变化

Climate Change from Large Language Models

School of Information Technology, Halmstad University(哈尔姆斯塔德大学信息技术学院)

Hongyin Zhu, Prayag Tiwari

AI总结:

本文提出自动化评估框架,通过混合数据获取和提示工程从10个视角评估大型语言模型的气候危机知识,发现其知识丰富但时效性不足。

摘要:

气候变化带来了严峻挑战,要求人们广泛理解并树立低碳生活意识。大型语言模型(LLMs)为应对这一危机提供了有力工具,但对其气候危机知识的全面评估仍然缺乏。本文提出了一种自动化评估框架,用于评估LLMs中的气候危机知识。我们采用数据获取的混合方法,结合数据合成与人工收集,编制了一套涵盖气候变化多个方面的多样化问题集。利用基于所编制问题的提示工程,我们通过分析模型生成的答案来评估其知识水平。此外,我们引入了一套全面的指标来评估气候危机知识,涵盖来自10个不同视角的指标。这些指标提供了多方面的评估,使我们能够细致地理解LLMs对气候危机的理解程度。实验结果证明了我们所提方法的有效性。在利用多种高性能LLMs进行的评估中,我们发现尽管LLMs拥有相当多的气候相关知识,但在时效性方面存在不足,这表明需要持续更新和完善其气候相关内容。

英文摘要:

Climate change poses grave challenges, demanding widespread understanding and low-carbon lifestyle awareness. Large language models (LLMs) offer a powerful tool to address this crisis, yet comprehensive evaluations of their climate-crisis knowledge are lacking. This paper proposes an automated evaluation framework to assess climate-crisis knowledge within LLMs. We adopt a hybrid approach for data acquisition, combining data synthesis and manual collection, to compile a diverse set of questions encompassing various aspects of climate change. Utilizing prompt engineering based on the compiled questions, we evaluate the model's knowledge by analyzing its generated answers. Furthermore, we introduce a comprehensive set of metrics to assess climate-crisis knowledge, encompassing indicators from 10 distinct perspectives. These metrics provide a multifaceted evaluation, enabling a nuanced understanding of the LLMs' climate crisis comprehension. The experimental results demonstrate the efficacy of our proposed method. In our evaluation utilizing diverse high-performing LLMs, we discovered that while LLMs possess considerable climate-related knowledge, there are shortcomings in terms of timeliness, indicating a need for continuous updating and refinement of their climate-related content.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑