释放潜力：大型语言模型在水工程与研究中的基准评测
Unlocking the Potential: Benchmarking Large Language Models in Water Engineering and Research
AI总结：
本文首次构建水工程领域基准WaterER，系统评测七个主流大模型在983项任务上的表现，揭示各模型优势并推动其成为真正的水专家。
摘要：
大型语言模型（LLMs）的最新进展引发了人们对其在各个领域潜在应用前景的兴趣。本文开展了一项关键探究：现有的LLMs能否有效充当水工程与研究任务中的“水专家模型”？本研究首次通过建立一个领域专用基准套件WaterER，评估了LLMs在各类水工程与研究任务中的贡献。在此，我们准备了983项与水工程和研究相关的任务，并将其划分为“污水处理”、“环境修复”、“饮用水处理与输配”、“环境卫生”、“厌氧消化”和“污染物评估”等类别。我们评估了七个LLMs（即GPT-4、GPT-3.5、Gemini、GLM-4、ERNIE、QWEN和Llama3）在这些任务上的表现。我们着重指出了GPT-4在处理水工程和水研究多样化复杂任务方面的优势，Gemini在学术语境中的专门能力，Llama3回答中文水工程问题的最强能力，以及GLM-4、ERNIE和QWEN等面向中文的模型在某些水工程任务中的竞争性表现。更具体地说，当前的LLMs尤其擅长为“污染物及相关水质监测与评估”方向的论文生成精确的研究空白。此外，它们更善于为“废水处理工艺”、“环境修复”和“饮用水处理”方向的研究论文拟定合适的标题。总体而言，本研究通过引入WaterER基准来评估其预测的可信度，开创了在水工程与研究领域对LLMs进行评测的先河。这一标准化评估框架还将通过使用针对性数据集推动LLM技术的未来发展，促使这些模型成为真正的“水专家”。
英文摘要：
Recent advancements in Large Language Models (LLMs) have sparked interest in their potential applications across various fields. This paper embarked on a pivotal inquiry: Can existing LLMs effectively serve as "water expert models" for water engineering and research tasks? This study was the first to evaluate LLMs' contributions across various water engineering and research tasks by establishing a domain-specific benchmark suite, namely, WaterER. Herein, we prepared 983 tasks related to water engineering and research, categorized into "wastewater treatment", "environmental restoration", "drinking water treatment and distribution", "sanitation", "anaerobic digestion" and "contaminants assessment". We evaluated the performance of seven LLMs (i.e., GPT-4, GPT-3.5, Gemini, GLM-4, ERNIE, QWEN and Llama3) on these tasks. We highlighted the strengths of GPT-4 in handling diverse and complex tasks of water engineering and water research, the specialized capabilities of Gemini in academic contexts, Llama3's strongest capacity to answer Chinese water engineering questions and the competitive performance of Chinese-oriented models like GLM-4, ERNIE and QWEN in some water engineering tasks. More specifically, current LLMs excelled particularly in generating precise research gaps for papers on "contaminants and related water quality monitoring and assessment". Additionally, they were more adept at creating appropriate titles for research papers on "treatment processes for wastewaters", "environmental restoration", and "drinking water treatment". Overall, this study pioneered evaluating LLMs in water engineering and research by introducing the WaterER benchmark to assess the trustworthiness of their predictions. This standardized evaluation framework would also drive future advancements in LLM technology by using targeting datasets, propelling these models towards becoming true "water expert".
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
