Digital art synthesis is receiving increasing attention in the multimedia community because of engaging the public with art effectively. Current digital art synthesis methods usually use single-modality inputs as guidance, thereby limiting the expressiveness of the model and the diversity of generated results. To solve this problem, we propose the multimodal guided artwork diffusion (MGAD) model, which is a diffusion-based digital artwork generation approach that utilizes multimodal prompts as guidance to control the classifier-free diffusion model. Additionally, the contrastive language-image pretraining (CLIP) model is used to unify text and image modalities. Extensive experimental results on the quality and quantity of the generated digital art paintings confirm the effectiveness of the combination of the diffusion model and multimodal guidance. Code is available at https://github.com/haha-lisa/MGAD-multimodal-guided-artwork-diffusion.
Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models
Shanghai Jiao Tong University（上海交通大学）；Institute of Artificial Intelligence, China Telecom (TeleAI)（中国电信人工智能研究院（TeleAI））；University of Science and Technology of China（中国科学技术大学）
DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）
DreamVideo: High-Fidelity Image-to-Video Generation with Image Retention and Text Guidance
Shenzhen Campus of Sun Yat-Sen University（中山大学深圳校区）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Shenzhen Campus of the Chinese University of Hong Kong（香港中文大学深圳校区）
DREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modeling
School of Cyber Science（网络安全学院）；State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室）；Zhejiang University（浙江大学）；Tsinghua University（清华大学）
MENTOR: Efficient Multimodal-Conditioned Tuning for Autoregressive Vision Generation Models
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Tsinghua University（清华大学）；Peking University（北京大学）；Microsoft（微软公司）
UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
School of Software Technology, Zhejiang University（浙江大学软件技术学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Computer Science（计算机科学学院）
DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editing
School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）；School of Computer Science, Peking University（北京大学计算机科学学院）；Huazhong University of Science and Technology（华中科技大学）
