MLLM 能否像人类一样进行批评？评估多模态大语言模型中的开放式审美推理
Can MLLMs Critique Like Humans? Evaluating Open-Ended Aesthetic Reasoning in Multimodal Large Language Models
AI总结：
本文评估多模态大语言模型在开放式审美批评中的表现，发现基于参考的相似度指标存在误导，模型在选择性、特异性和多样性方面与人类批评存在系统性差异。
摘要：
开放式审美批评对多模态大语言模型（MLLM）是一个挑战：与多项选择的审美基准不同，它没有单一正确答案，且大多数审美评估通过数值评分而非人类实际给出的书面批评来衡量模型。我们基于排名的人类参考评估MLLM的批评，并询问它们是否接近人类批评。使用Reddit照片批评数据集，我们通过基于参考的相似度指标，在六种提示条件下（解耦角色框架、方面提示、长度控制、单次与多次生成，并添加图像基础控制，即向每个模型输入错误的照片）对五个开放权重的MLLM进行评分。我们发现基于参考的相似度给出了误导性图像。更严格的词汇和习得指标仅显示与人类批评的弱对齐，而粗略的嵌入余弦报告了广泛的主题重叠，基础控制将其追溯到稳定的风格而非特定图像的观察。在行为上，模型以分数未揭示的一致方式与人类偏离：即使在长度限制下，它们写的量是人类的2到3倍，覆盖了人类选择性的几乎所有审美方面，更均匀且更深入地参与每个方面，并在同一张照片的批评中重复自己，而人类则变化多样。我们认为基于参考的相似度奖励流畅、全面的批评风格，而非人类批评的选择性和特异性，并讨论了评估和训练开放式多模态生成的含义。
英文摘要：
Open-ended aesthetic critique is a challenge for multimodal large language models (MLLMs): it has no single correct answer, and most aesthetic evaluation measures models against numeric scores rather than the written critiques people actually give. We ask whether MLLM critiques are close to human ones, scoring eight open-weight MLLMs from $7$B to $397$B, plus GPT-5.5, against multiple ranked human critiques for each of $1{,}227$ \texttt{r/photocritique} posts under eight prompt conditions. Reference-based similarity gives a misleading picture. In absolute terms the stricter lexical and learned metrics align only weakly with human critiques while a coarse embedding cosine reports broad topical overlap, yet requesting shorter critiques raises those scores and withholding the image barely changes them: the similarity reflects length, the post text, and a stable critiquing style more than image-specific observation. An LLM judge sharpens the question rather than settling it: in the primary condition all four judges prefer the frontier models' critiques to the human ones, but on the $7$--$8$B models they diverge wildly, from $9\%$ to $81\%$ preference on identical pairs. Asked instead how similar each pair is in substance, those judges and two human annotators agree, rating every model between $1.81$ and $2.59$ on a $1$--$5$ scale, close to ``mostly different''. Behaviorally, the models diverge in ways the scores do not surface: they cover nearly every aesthetic aspect where humans are selective and repeat themselves across critiques of one photo, even when prompted to write at human length. We argue that reference-based similarity rewards a fluent, comprehensive critique style rather than the selectivity and specificity of human critique.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
