arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.06170cs.CV
收藏

DynT2I-Eval: 一种用于文本到图像模型的动态评估框架

DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models

Institute of Image Communication and Network Engineering(图像通信与网络工程研究所)

Juntong Wang, Jiarui Wang, Huiyu Duan, Lewei Li, Guangtao Zhai, Xiongkuo Min

AI总结:

本文提出DynT2I-Eval框架,通过动态生成提示以提升文本到图像模型的评估鲁棒性,减少过拟合和基准污染影响。

摘要:

现有文本到图像(T2I)基准主要依赖固定提示集,一旦公开和重复使用,容易导致过拟合和基准污染。本文提出DynT2I-Eval,一种完全自动化的动态评估框架,从长形式描述构建结构化的视觉语义空间,将提示分解为可控维度(如主体、逻辑约束、环境和构图)。这使任务特定空间和难度感知采样能够持续生成新鲜提示。DynT2I-Eval评估模型在文本对齐、感知质量和美学方面的能力。异构输出被统一为提示条件下的成对比较,允许动态调度器、微批次聚合和加权贝叶斯更新,以维持稳定在线排行榜,即使在提示分布和模型注入变化时。实验表明,持续刷新的提示提供稳健的评估协议,减少提示集特定调优的影响。模拟和消融进一步证实,所提出的排名框架在冷启动收敛、后期进入发现和长期排名保真度之间实现了强平衡。

英文摘要:

Existing text-to-image (T2I) benchmarks largely rely on fixed prompt sets, leaving them vulnerable to overfitting and benchmark contamination once publicly released and repeatedly reused. In this work, we propose DynT2I-Eval, a fully automated dynamic evaluation framework for T2I models. It constructs a structured visual semantic space from long-form descriptions, decomposing prompts into controllable dimensions (e.g., subject, logical constraint, environment, and composition). This enables the continuous generation of fresh prompts via task-specific spaces and difficulty-aware sampling. DynT2I-Eval evaluates model performance across text alignment, perceptual quality, and aesthetics. Heterogeneous outputs are unified into prompt-conditioned pairwise comparisons, allowing a dynamic scheduler, micro-batch aggregation, and weighted Bayesian updates to maintain a stable online leaderboard despite changing prompt distributions and model injection. Experiments with independently sampled prompt streams demonstrate that continually refreshed prompts provide a robust evaluation protocol, reducing the impact of prompt-set-specific tuning. Simulations and ablations further confirm that the proposed ranking framework achieves a strong balance among cold-start convergence, late-entry discovery, and long-run ranking fidelity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑