学习规则增强的大语言模型评估器
Learned-Rule-Augmented Large Language Model Evaluators
AI总结：
本文提出规则增强的LLM评估器，通过规则蒸馏和强化学习提升评估器的泛化能力与一致性，适用于多种评估场景。
摘要：
大型语言模型（LLMs）主要用于自然语言生成（NLG）任务的评估，但其在更广泛评估场景中的应用仍有限。在本工作中，我们探索了LLMs作为跨多样任务的通用评估器的潜力。尽管基于LLM的评估器在不同领域取得了进展，但现有方法在泛化方面存在困难，因为它们依赖于昂贵的人工设计评估原则，这些原则往往与标注数据和LLM的理解不一致。为了解决这些挑战，我们提出了一种规则增强的评估范式。首先，我们介绍了一种规则蒸馏方法，该方法利用LLM辅助的蒙特卡洛树搜索（MCTS）自动从数据中提取评分规则，从而缓解可扩展性问题并提高与数据的一致性。其次，为了使LLM有效应用学习到的规则，我们提出了两种策略：（1）规则链（CoR），它指导LLM遵循蒸馏的规则，以及（2）通过强化学习训练规则增强的LLM评估器（RuAE），进一步弥合规则与LLM推理之间的差距。在各种任务上的广泛实验表明，我们的方法在各种评估场景中都表现出有效的效果和泛化能力。
英文摘要：
Large language models (LLMs) are predominantly used as evaluators for natural language generation (NLG) tasks, but their application to broader evaluation scenarios remains limited. In this work, we explore the potential of LLMs as general evaluators across diverse tasks. Although LLM-based evaluators have made progress in different areas, existing methods struggle to generalize due to their reliance on costly, human-designed evaluation principles, which are often misaligned with both annotated data and LLMs' understanding.To address these challenges, we propose a rule-augmented evaluation paradigm. First, we introduce a rule distillation method that automatically extracts scoring rules from data using an LLM-assisted Monte Carlo Tree Search (MCTS), alleviating scalability issues and improving alignment with data. Second, to enable LLMs to effectively apply the learned rules, we propose two strategies: (1) Chain-of-Rule (CoR), which guides LLM to follow distilled rules, and (2) training a rule-augmented LLM evaluator (RuAE) via reinforcement learning, further bridging the gap between rules and LLMs' reasoning. Extensive experiments on diverse tasks demonstrate the effectiveness and generalizability of our approach across various evaluation scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
