arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.06277cs.CLcs.IR
收藏

环境大语言模型评估(ELLE)数据集:面向生态环境领域生成式AI应用的评估基准

Environmental large language model Evaluation (ELLE) dataset: A Benchmark for Evaluating Generative AI applications in Eco-environment Domain

Beijing Information Science & Technology University(北京信息科技大学);Tsinghua University(清华大学)

Jing Guo, Nan Li, Ming Xu

AI总结:

针对生态环境领域生成式AI缺乏统一评估框架的问题,研究团队推出首个该领域专用的ELLE问答基准数据集,含16个主题的1130组问答对,可规范评估、推动生成式AI助力环境可持续发展。

摘要:

生成式AI在生态环境监测、数据分析、教育与政策支持等场景具备巨大应用潜力,但受限于缺乏统一评估框架,其应用效果受到制约。为此,我们提出环境大语言模型评估(ELLE)问答数据集,这是首个专门用于评估大语言模型及其在生态环境科学领域应用的基准。该数据集包含覆盖16个环境主题的1130组问答对,按领域、难度和类型进行分类。这套全面的数据集可规范该领域的性能评估流程,实现对生成式AI性能一致、客观的比较。通过提供专用评估工具,ELLE数据集将推动生成式AI技术在实现可持续环境目标方面的发展与应用。数据集与代码可访问https://elle.ceeai.net/ 和https://github.com/CEEAI/elle 获取。

英文摘要:

Generative AI holds significant potential for ecological and environmental applications such as monitoring, data analysis, education, and policy support. However, its effectiveness is limited by the lack of a unified evaluation framework. To address this, we present the Environmental Large Language model Evaluation (ELLE) question answer (QA) dataset, the first benchmark designed to assess large language models and their applications in ecological and environmental sciences. The ELLE dataset includes 1,130 question answer pairs across 16 environmental topics, categorized by domain, difficulty, and type. This comprehensive dataset standardizes performance assessments in these fields, enabling consistent and objective comparisons of generative AI performance. By providing a dedicated evaluation tool, ELLE dataset promotes the development and application of generative AI technologies for sustainable environmental outcomes. The dataset and code are available at https://elle.ceeai.net/ and https://github.com/CEEAI/elle.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑