领域精通基准：一个持续更新的大语言模型整体领域知识评估基准——初步发布
Domain Mastery Benchmark: An Ever-Updating Benchmark for Evaluating Holistic Domain Knowledge of Large Language Model--A Preliminary Release
AI总结：
本文提出DomMa基准，基于中国112个一级学科构建10万道中英文题目，用于持续、全面地评估大语言模型的领域知识理解能力。
摘要：
领域知识指对特定学科、行业、领域或专门兴趣方向的深入理解、专业知识与熟悉程度。现有基准均缺乏对领域知识评估的整体设计。我们认为，只有通过全面且深入的基准，才能公平评估领域语言理解的真实能力，因此提出了Domma，即领域精通基准。DomMa旨在测试大语言模型（LLMs）的领域知识理解能力，具有广泛的领域覆盖、大规模数据量，并基于中国112个一级学科分类持续更新数据集。DomMa包含100,000道中英文题目，来源于中国高校的研究生入学考试和本科考试。我们还提出了使基准和评估过程更适合大语言模型的设计方案。
英文摘要：
Domain knowledge refers to the in-depth understanding, expertise, and familiarity with a specific subject, industry, field, or area of special interest. The existing benchmarks are all lack of an overall design for domain knowledge evaluation. Holding the belief that the real ability of domain language understanding can only be fairly evaluated by an comprehensive and in-depth benchmark, we introduces the Domma, a Domain Mastery Benchmark. DomMa targets at testing Large Language Models (LLMs) on their domain knowledge understanding, it features extensive domain coverage, large data volume, and a continually updated data set based on Chinese 112 first-level subject classifications. DomMa consist of 100,000 questions in both Chinese and English sourced from graduate entrance examinations and undergraduate exams in Chinese college. We have also propose designs to make benchmark and evaluation process more suitable to LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
