Text-to-image generation has recently emerged as a viable alternative to text-to-image retrieval, driven by the visually impressive results of generative diffusion models. Although query performance prediction is an active research topic in information retrieval, to the best of our knowledge, there is no prior study that analyzes the difficulty of queries (referred to as prompts) in text-to-image generation, based on human judgments. To this end, we introduce the first dataset of prompts which are manually annotated in terms of image generation performance. Additionally, we extend these evaluations to text-to-image retrieval by collecting manual annotations that represent retrieval performance. We thus establish the first joint benchmark for prompt and query performance prediction (PQPP) across both tasks, comprising over 10K queries. Our benchmark enables (i) the comparative assessment of prompt/query difficulty in both image generation and image retrieval, and (ii) the evaluation of prompt/query performance predictors addressing both generation and retrieval. We evaluate several pre- and post-generation/retrieval performance predictors, thus providing competitive baselines for future research. Our benchmark and code are publicly available at https://github.com/Eduard6421/PQPP.
Think Bright, Diffuse Nice: Enhancing T2I-ICL via Inductive-Bias Hint Instruction and Query Contrastive Decoding
Cao Tu Li (Guangzhou) Technology Co., Ltd（曹图利（广州）科技有限公司）；South China University of Technology（华南理工大学）；Washington State University（华盛顿州立大学）；Hong Kong Baptist University（香港 Baptist 大学）
DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Drexel University（德雷塞尔大学）；Shanghai Innovation Center for Processor Technologies, SHIC（上海处理器技术创新中心）
Adaptive Query Prompting for Multi-Domain Landmark Detection
School of Computer Science and Technology, Guangdong University of Technology（广东技术大学计算机科学与技术学院）；School of Automation, Guangdong University of Technology（广东技术大学自动化学院）；Faculty of Applied Sciences, Macao Polytechnic University（澳门 polytechnic university 应用科学学院）；School of Information Science and Technology, Guangdong University of Foreign Studies（广东外语外贸大学信息科学与技术学院）；School of Computer Science and Engineering, Huizhou University（惠州市大学计算机科学与工程学院）
CRISP: Contrastive Residual Injection and Semantic Prompting for Continual Video Instance Segmentation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences（机器人与智能系统国家重点实验室，沈阳自动化研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）
Love Me, Love My Label: Rethinking the Role of Labels in Prompt Retrieval for Visual In-Context Learning
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）
FaVChat: Hierarchical Prompt-Query Guided Facial Video Understanding with Data-Efficient GRPO
State Key Laboratory of Networking and Switching Technology, BUPT（北京邮电大学网络与交换技术国家重点实验室）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室（深圳））；Institute of automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Zhongguancun Academy（中关村学院）；Central South University（中南大学）；Zhejiang University（浙江大学）；College of communication Engineering, Hangzhou Dianzi University（杭州电子科技大学通信工程学院）
FreestyleRet: Retrieving Images from Style-Diversified Queries
Peking University Shenzhen Graduate School（北京大学深圳研究生院）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Peng Cheng Laboratory（鹏城实验室）；Tsinghua University（清华大学）
Bi-directional Training for Composed Image Retrieval via Text Prompt Learning
Australian National University（澳大利亚国立大学）；Idiap Research Institute（Idiap 研究所）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）
