Beyond Text Prompts: Visual-to-Visual Generation as A Unified Paradigm
City University of Hong Kong(香港城市大学);City University of Hong Kong (Dongguan)(香港城市大学(东莞));The Hong Kong University of Science and Technology(香港科学与技术大学);The Chinese University of Hong Kong(香港中文大学);Celia Research HK(Celia研究香港);Great Bay University(大湾大学);Lingnan University(岭南大学)
Yaofang Liu, Kangning Cui, Meng Chu, Zhaoqing Li, Suiyun Zhang, Jean-Michel Morel, Xiaodong Cun, Haoxuan Che, Rui Liu, Raymond H. Chan
Humans often specify and create through visual artifacts: typography sheets, sketches, reference images, and annotated scenes. Yet modern visual generators still ask users to serialize this intent into text, a bottleneck that compresses signals like spatial structure, exact appearance, and glyph shape. We propose \textbf{\emph{visual-to-visual} (V2V)} generation, in which the user conditions a generative model with a visual specification page rather than a text prompt. The page is not an edit target, but a visual document that specifies the desired output. We introduce \textbf{V2V-Zero}, a training-free framework that exposes this interface in existing vision-language model (VLM) conditioned generators by replacing text-only conditioning with final-layer hidden states extracted from visual pages, exploiting the fact that the frozen VLM already maps both text and images into the generator's conditioning space.
On GenEval, V2V-Zero reaches 0.85 with a frozen Qwen-Image backbone, closely matching its optimized text-to-image performance without fine-tuning. To evaluate the broader V2V space, we introduce \textbf{Simple-V2V Bench}, spanning seven visual-conditioning tasks and seven models, including GPT Image 2, Nano Banana 2, Seedream 5.0 Lite, open-weight baselines, and a video extension. V2V-Zero scores 32.7/100, outperforming evaluated open-weight image baselines and revealing a clear capability hierarchy: attribute binding is strong, content generation is unreliable, and structural control remains hard even for commercial systems. A HunyuanVideo-1.5 extension scores 20.2/100, showing the interface transfers beyond images. Mechanistic analysis shows the default reasoning path is primarily visually routed, with 95.0\% of conditioning-token attention mass on visual-page hidden states.
Causality-guided Prompt Learning for Vision-language Models via Visual Granulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学)
EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing
Advanced Research Institute of Multidisciplinary Sciences, Beijing Institute of Technology(北京理工大学多学科先进研究院);School of Mechatronical Engineering, Beijing Institute of Technology(北京理工大学机电工程学院);Yangtze Delta Region Academy of Beijing Institute of Technology(北京理工大学长三角研究院)
FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Models
University of Trento(特伦托大学);University of Bergamo(贝拉姆奥大学);Indian Institute of Technology Bombay(印度班加罗尔理工学院);LNMIIT Jaipur(斋普尔LNMIIT);The University of Queensland(昆士兰大学);Fondazione Bruno Kessler(布鲁诺·凯塞勒基金会)
Visual Prompting in Multimodal Large Language Models: A Survey
UC San Diego(加州大学圣地亚哥分校);Dartmouth College(达特茅斯学院);Rutgers University(罗格斯大学);UC Los Angeles(加州大学洛杉矶分校);Adobe Research(奥多比研究院);The University of New South Wales(新南威尔士大学);CSIRO’s Data61(联邦科学与工业研究组织Data61)
Guiding Medical Vision-Language Models with Explicit Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations
Sichuan University Pittsburgh Institute(四川大学匹兹堡学院);Beijing University of Posts and Telecommunications(北京邮电大学);Brown University(布朗大学);Case Western Reserve University(凯斯西储大学);Shanghai Jiao Tong University(上海交通大学)
DualCap: Enhancing Lightweight Image Captioning via Dual Retrieval with Similar Scenes Visual Prompts
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering
Khoury College of Computer Sciences, Northeastern University(东北大学克劳尔计算机科学学院);New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别新实验室)
A Similarity Paradigm Through Textual Regularization Without Forgetting
Shanghai Jiao Tong University(上海交通大学);Hong Kong Baptist University(香港浸会大学);Northeastern University(东北大学);University of Science and Technology of China(中国科学技术大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM
Zhejiang University(浙江大学);Shanghai AI lab(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);The University of Sydney(悉尼大学);University of Oxford(牛津大学)
Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models
School of Engineering, Westlake University(西湖大学工程学院);City University of Hong Kong(香港城市大学);Research Center for Industries of the Future, Westlake University(西湖大学未来产业研究中心);Johns Hopkins University(约翰斯·霍普金斯大学)
Prompt-OT: An Optimal Transport Regularization Paradigm for Knowledge Preservation in Vision-Language Model Adaptation
Morgan Stanley(摩根士丹利);Clemson University(克莱姆森大学);Arizona State University(亚利桑那州立大学);Washington University in St. Louis(圣路易斯华盛顿大学);University of Arizona(亚利桑那大学);University of Notre Dame(圣母大学)
Exploring Interpretability for Visual Prompt Tuning with Cross-layer Concepts
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);Microsoft Research Asia(微软亚洲研究院);School of Telecommunication and Engineering, Xidian University(西安电子科技大学电信与工程学院)
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));vivo Mobile Communication Co., Ltd.(维沃移动通信有限公司);Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所);Pengcheng Laboratory(鹏城实验室)
CP3: Unifying Point Cloud Completion by Pretrain-Prompt-Predict Paradigm
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Shanghai AI Laboratory(上海人工智能实验室)
Multiple Queries with Multiple Keys: A Precise Prompt Matching Paradigm for Prompt-based Continual Learning
National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);School of Electronic Science and Engineering, Nanjing University(电子科学与工程学院,南京大学)
LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation
Rutgers University(罗格斯大学);Zhejiang University(浙江大学);Red Hat AI Innovation(红帽人工智能创新中心);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);NVIDIA Research(英伟达研究院)
VisualPrompter: Semantic-Aware Prompt Optimization with Visual Feedback for Text-to-Image Synthesis
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Academy of Artificial Intelligence(北京人工智能研究院);University of Chinese Academy of Sciences(中国科学院大学);Peking University(北京大学)