Text-to-image (T2I) models struggle with compositional prompts requiring multiple objects, bound attributes, accurate counts, or spatial relations. Existing benchmarks face limits from fixed detector vocabularies, aggregated scores, and varying human-validation protocols. We introduce SANEval (Spatial, Attribute, and Numeracy Evaluation), an open-vocabulary compositional benchmark with two key contributions: (i) a public dataset of ~5,250 prompts paired with images from six state-of-the-art T2I models, split into algorithmically generated Simple and fully human-written Hard partitions across five compositional tasks, and (ii) a modular evaluation pipeline combining an LLM prompt parser, an open-vocabulary detector with LLM synonym expansion, and scorers for attribute binding, spatial relations, and numeracy. Each scorer emits numerical scores and diagnostic feedback for missing, extra, or mis-bound elements. Validated against human ratings (n = 500), SANEval achieves positive Pearson correlations across all categories (overall r = 0.222), outperforming CompBench++ (-0.016) and GenEval (-0.030). A manual audit of 24,012 detections shows 96.58% precision, with ablation confirming that synonym expansion is critical to pipeline performance. Dataset: [https://huggingface.co/datasets/saneval/saneval-release](https://huggingface.co/datasets/saneval/saneval-release). Pipeline: [https://github.com/nielseni6/saneval](https://github.com/nielseni6/saneval).
Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection
Zhejiang University（浙江大学）；The Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）；Singapore Management University（新加坡管理大学）
Decomposed Vision-Language Alignment for Fine-Grained Open-Vocabulary Segmentation
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空航天信息研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhejiang University（浙江大学）
