双层提示优化用于多模态LLM作为裁判
Bi-Level Prompt Optimization for Multimodal LLM-as-a-Judge
AI总结：
本文提出BLPO框架，通过双层优化提升多模态LLM在评估AI生成图像时的提示效果，解决上下文限制导致的优化瓶颈问题。
摘要：
大型语言模型（LLM）已广泛应用于自动评估AI生成内容的裁判。尽管其成功，但将LLM基于的评估与人类判断对齐仍然具有挑战性。虽然在人工标注数据上进行监督微调可以提高对齐性，但这种方法成本高且不灵活，需要为每个任务或数据集重新训练。最近在自动提示优化（APO）方面的进展提供了一种更高效的替代方法，通过自动改进指导LLM裁判的指令。然而，现有APO方法主要针对文本-only评估，在多模态设置中仍处于探索阶段。在本工作中，我们研究了多模态LLM作为裁判的自动提示优化，特别是用于评估AI生成图像。我们发现一个关键瓶颈：多模态模型由于上下文窗口限制，只能处理有限数量的视觉示例，这阻碍了有效的试错提示优化。为克服这一问题，我们提出了BLPO，一种双层提示优化框架，将图像转换为文本表示的同时保留与评估相关的视觉线索。我们的双层优化方法联合优化裁判提示和I2T提示，以在有限的上下文预算下保持保真度。在四个数据集和三个LLM裁判上的实验展示了我们方法的有效性。
英文摘要：
Large language models (LLMs) have become widely adopted as automated judges for evaluating AI-generated content. Despite their success, aligning LLM-based evaluations with human judgments remains challenging. While supervised fine-tuning on human-labeled data can improve alignment, it is costly and inflexible, requiring new training for each task or dataset. Recent progress in auto prompt optimization (APO) offers a more efficient alternative by automatically improving the instructions that guide LLM judges. However, existing APO methods primarily target text-only evaluations and remain underexplored in multimodal settings. In this work, we study auto prompt optimization for multimodal LLM-as-a-judge, particularly for evaluating AI-generated images. We identify a key bottleneck: multimodal models can only process a limited number of visual examples due to context window constraints, which hinders effective trial-and-error prompt refinement. To overcome this, we propose BLPO, a bi-level prompt optimization framework that converts images into textual representations while preserving evaluation-relevant visual cues. Our bi-level optimization approach jointly refines the judge prompt and the I2T prompt to maintain fidelity under limited context budgets. Experiments on four datasets and three LLM judges demonstrate the effectiveness of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
