发表机构
AI Lab, Qifu Technology; Beijing University of Posts and Telecommunications; University of Science and Technology Beijing; Tongji University（奇富科技AI实验室; 北京邮电大学; 北京科技大学; 同济大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
CriticGen提出生成感知的细粒度评估框架，通过动态评分细则生成可操作反馈，显著提升评估相关性和答案改进率。
AI 中文摘要
当前大型语言模型的评估方法粒度较粗且与生成过程脱节，产生的解释过于笼统，无法为模型改进提供可操作的反馈。我们提出CriticGen，一种细粒度、生成感知的评估框架，将评估转化为对答案改进的可操作控制。CriticGen首先在主观、客观和自衍生约束等高级类别下生成样本特定的评估维度和评分标准。这些标准随后作为动态评分细则，用于联合生成分数、理由、可执行的改进建议以及改进后的答案。这种基于评分细则的条件化改进过程使模型能够诊断缺陷并进行有针对性的答案改进。实验结果表明，细粒度评估应既具有实例特定性又具有可操作性。CriticGen生成更高质量的评分细则，将相关性/覆盖率从3.33/4.03提升至3.97/4.24。CriticGen还实现了最佳分数相关性，皮尔逊相关系数为0.9556，斯皮尔曼相关系数为0.9560，并将基于标准的理由和可执行建议的F1分数从0.6369/0.5994提升至0.7554/0.7900。关键在于，其反馈转化为可靠的答案改进，改进了73.17%的答案，且非退化率为93.28%。
英文摘要
Current evaluation methods for large language models are coarse-grained and decoupled from generation, producing generic explanations that fail to provide actionable feedback for model improvement. We propose CriticGen, a fine-grained, generation-aware evaluation framework that turns evaluation into actionable control for answer improvement. CriticGen first generates sample-specific evaluation dimensions and scoring criteria under high-level categories such as subjective, objective, and self-derived constraints. These criteria then serve as a dynamic rubric for jointly producing a score, a reason, an executable refinement suggestion, and a refined answer. This rubric-conditioned refinement process enables models to diagnose flaws and perform targeted answer improvement. Experimental results show that fine-grained evaluation should be both instance-specific and actionable. CriticGen induces higher-quality rubrics, improving relevance/coverage from 3.33/4.03 to 3.97/4.24. CriticGen also achieves the best score correlations, with 0.9556 Pearson and 0.9560 Spearman, and raises the F1 of criterion-grounded reasons and executable suggestions from 0.6369/0.5994 to 0.7554/0.7900. Crucially, its feedback translates into reliable answer improvement, improving 73.17% of answers with a 93.28% non-degradation rate.
