Manipulating the material appearance of objects in images is critical for applications like augmented reality, virtual prototyping, and digital content creation. We present MaterialFusion, a novel framework for high-quality material transfer that allows users to adjust the degree of material application, achieving an optimal balance between new material properties and the object's original features. MaterialFusion seamlessly integrates the modified object into the scene by maintaining background consistency and mitigating boundary artifacts. To thoroughly evaluate our approach, we have compiled a dataset of real-world material transfer examples and conducted complex comparative analyses. Through comprehensive quantitative evaluations and user studies, we demonstrate that MaterialFusion significantly outperforms existing methods in terms of quality, user control, and background preservation. Code is available at https://github.com/ControlGenAI/MaterialFusion.
Diff-PC: Identity-preserving and 3D-aware Controllable Diffusion for Zero-shot Portrait Customization
School of Electronic Science and Engineering, Nanjing University(南京大学电子科学与工程学院);School of Business, Hohai University(河海大学商学院);School of Artificial Intelligence, Nanjing University of Information Science and Technology(南京信息工程大学人工智能学院)
Visual Prompting for One-shot Controllable Video Editing without Inversion
Singapore University of Technology and Design(新加坡科技设计大学);Wuhan University(武汉大学);Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore(新加坡资讯与通信研究院)
ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editing
Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));Pengcheng Laboratory, China(鹏城实验室);Harbin Institute of Technology, China(哈尔滨工业大学);Harbin Institute of Technology, Suzhou Research Institute, China(哈尔滨工业大学苏州市研究院)
DivControl: Knowledge Diversion for Controllable Image Generation
School of Computer Science and Engineering, Southeast University, Nanjing, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其跨学科应用国家重点实验室(东南大学))
DreamShot: Personalized Storyboard Synthesis with Video Diffusion Prior
Sun Yat-sen University(中山大学);Peng Cheng Laboratory(鹏城实验室);ByteDance Intelligent Creation(字节跳动智能创作);Guangdong Key Laboratory of Big Data Analysis and Processing(广东省大数据分析与处理重点实验室)
MS-CustomNet: Controllable Multi-Subject Customization with Hierarchical Relational Semantics
School of Information Science and Engineering, East China University of Science and Technology(信息科学与工程学院,东华大学);Center for Applied Mathematics, Tianjin University(应用数学中心,天津大学)
Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling
The Chinese University of Hong Kong(香港中文大学);SenseTime Research(商汤科技研究院);NVIDIA AI Technology Center(英伟达人工智能技术中心);Tsinghua University(清华大学);Centre for Perceptual and Interactive Intelligence (CPII)(感知与交互智能中心);Shanghai AI Laboratory(上海人工智能实验室);Avolution AI
OPT: One-shot Pose-Controllable Talking Head Generation
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 2 School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China(1 中国科学院信息工程研究所,北京,中国 2 中国科学院大学网络空间安全学院,北京,中国)
Non-generative Generalized Zero-shot Learning via Task-correlated Disentanglement and Controllable Samples Synthesis
Beijing Jiaotong University(北京交通大学)
伪样本合成是目前解决广义零样本学习问题的最有效方法。大多数模型都取得了较好的性能,但仍然存在两个问题:(1)特征混淆,总体表示混淆了任务相关和任务无关的特征,现有模型以生成的方式将它们分离,但在有限样本的情况下合成可靠的伪样本是不合理的;(2)分布的不确定性,现有的模型从不确定的分布中合成样本时需要大量的数据,这导致在有限的可见类样本中性能不佳。本文提出了一个非生成式的模型来解决这些问题,该模型分为两个模块:(1)任务相关特征去纠缠,通过领域适应的对抗性学习,将任务相关特征从任务无关特征中排除,实现合理的综合;(2)可控伪样本合成,以合成具有某些特性的边缘伪样本和中心伪样本,从而产生更多的多样性和直观的传递。此外,为了描述训练过程中出现的有限可见类样本这一新场景,我们进一步提出了一种新的可重构学习任务,称为“Few-Shot Seen class and Zero-shot Unseen class learning”(FSZU)。在四个基准测试上的大量实验验证了所提出的方法在GALML和FSZU任务中具有竞争力。
HUG-VAS: A Hierarchical NURBS-Based Generative Model for Aortic Geometry Synthesis and Controllable Editing
Department of Aerospace and Mechanical Engineering, University of Notre Dame(notre dame 大学航空航天与机械工程系);Sibley School of Mechanical and Aerospace Engineering, Cornell University(cornell 大学机械与航空航天工程学院);Department of Applied and Computational Mathematics and Statistics, University of Notre Dame(notre dame 大学应用与计算数学与统计学系)
TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models
University of Hong Kong(香港大学);University College London(伦敦大学学院);University of Alberta(阿尔伯塔大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Zhejiang University(浙江大学)