超越一刀切：用于高效NLG评估提示的反向学习
Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts
AI总结：
针对NLG评估提示敏感性问题，提出反向学习方法，自动生成模型特定评估提示，仅需单样本，提升效率与鲁棒性。
摘要：
评估自然语言生成系统因有效输出的多样性而具有挑战性。尽管人工评估是黄金标准，但它存在不一致性、缺乏标准化以及人口统计学偏见等问题，限制了可复现性。基于LLM的评估器提供了一种可扩展的替代方案，但对提示设计高度敏感，微小的变化可能导致显著差异。在这项工作中，我们提出了一种反向学习方法，学习从模型输出回到其输入指令的有效反向映射，从而能够自动生成高度有效、针对特定模型的评估提示。我们的方法仅需单个评估样本，无需耗时的提示人工设计，从而提高了效率和鲁棒性。我们的工作为更鲁棒和高效的基于LLM的评估开辟了新方向。
英文摘要：
Evaluating natural language generation systems is challenging due to the diversity of valid outputs. While human evaluation is the gold standard, it suffers from inconsistencies, lack of standardisation, and demographic biases, limiting reproducibility. LLM-based evaluators offer a scalable alternative but are highly sensitive to prompt design, where small variations can lead to significant discrepancies. In this work, we propose an inversion learning method that learns effective reverse mappings from model outputs back to their input instructions, enabling the automatic generation of highly effective, model-specific evaluation prompts. Our method requires only a single evaluation sample and eliminates the need for time-consuming manual prompt engineering, thereby improving both efficiency and robustness. Our work contributes toward a new direction for more robust and efficient LLM-based evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
