arXiv 2507.229112026-01-30cs.CLcs.AI

ElectriQ：一个评估大型语言模型在电力营销中响应能力的基准

ElectriQ: A Benchmark for Assessing the Response Capability of Large Language Models in Power Marketing

Systems Engineering Institute, Xi’an Jiaotong University（系统工程研究所，西安交通大学）；State Key Laboratory of Multiphase Flow in Power Engineering, School of Energy and Power Engineering, Xi’an Jiaotong University（电力工程多相流国家重点实验室，能源与动力工程学院，西安交通大学）；School of Electronic Science and Engineering, Xi'an Jiaotong University（电子科学与工程学院，西安交通大学）；Organizational Management Department, School of Management, Xi’an Jiaotong University（组织管理部，管理学院，西安交通大学）

Jinzhi Wang, Qingke Peng, Haozhou Li, Zeyuan Zeng, Jiangbo Zhang, Kaixuan Yang, Ningyong Wu, Qinfeng Song, Ruimeng Li, Biyi Zhou

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

ElectriQ是一个用于评估大型语言模型在电力营销中响应能力的基准，通过结合人类评分、自动指标和合规测试，提出SEEK-RAG方法提升领域知识注入，实现高效且合规的电力营销助手部署。

摘要：

随着电力系统去碳化和数字化，分布式能源资源的高渗透率和灵活电价使电力营销（EPM）成为连接监管、系统运行和可持续能源部署的关键接口。许多公用事业公司仍依赖于人类代理和基于规则或意图的聊天机器人，这些机器人拥有碎片化的知识库，在处理长对话和跨场景对话时遇到困难，无法满足合规、可验证和DR（需求响应）准备的交互要求。同时，前沿的大语言模型（LLMs）展现出强大的对话能力，但它们的评估基于通用基准，这些基准对行业特定术语、监管推理和多轮对话稳定性重视不足。为了解决这一差距，我们提出了ElectriQ，一个用于电力营销中LLM的大型基准和评估框架。ElectriQ包含超过550,000个对话，涵盖六个服务领域和24个子场景，并定义了一个统一的协议，结合了人类评分、自动指标和两个合规压力测试——法定引用正确性和长对话一致性。基于ElectriQ，我们提出了一种检索增强的方法SEEK-RAG，该方法在微调和推理过程中注入政策和领域知识。在13个LLM上的实验表明，与领域对齐的7B模型结合SEEK-RAG可以匹配或超越更大模型，同时减少计算成本，为部署支持需求侧管理、可再生能源整合和电网韧性的LLM电力营销助手提供了可审计、监管意识的基础。

英文摘要：

As power systems decarbonise and digitalise, high penetrations of distributed energy resources and flexible tariffs make electric power marketing (EPM) a key interface between regulation, system operation and sustainable-energy deployment. Many utilities still rely on human agents and rule- or intent-based chatbots with fragmented knowledge bases that struggle with long, cross-scenario dialogues and fall short of requirements for compliant, verifiable and DR-ready interactions. Meanwhile, frontier large language models (LLMs) show strong conversational ability but are evaluated on generic benchmarks that underweight sector-specific terminology, regulatory reasoning and multi-turn process stability. To address this gap, we present ElectriQ, a large-scale benchmark and evaluation framework for LLMs in EPM. ElectriQ contains over 550k dialogues across six service domains and 24 sub-scenarios and defines a unified protocol that combines human ratings, automatic metrics and two compliance stress tests-Statutory Citation Correctness and Long-Dialogue Consistency. Building on ElectriQ, we propose SEEK-RAG, a retrieval-augmented method that injects policy and domain knowledge during finetuning and inference. Experiments on 13 LLMs show that domain-aligned 7B models with SEEK-RAG match or surpass much larger models while reducing computational cost, providing an auditable, regulation-aware basis for deploying LLM-based EPM assistants that support demand-side management, renewable integration and resilient grid operation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。