We address the challenges of precise image inversion and disentangled image editing in the context of few-step diffusion models. We introduce an encoder based iterative inversion technique. The inversion network is conditioned on the input image and the reconstructed image from the previous step, allowing for correction of the next reconstruction towards the input image. We demonstrate that disentangled controls can be easily achieved in the few-step diffusion model by conditioning on an (automatically generated) detailed text prompt. To manipulate the inverted image, we freeze the noise maps and modify one attribute in the text prompt (either manually or via instruction based editing driven by an LLM), resulting in the generation of a new image similar to the input image with only one attribute changed. It can further control the editing strength and accept instructive text prompt. Our approach facilitates realistic text-guided image edits in real-time, requiring only 8 number of functional evaluations (NFEs) in inversion (one-time cost) and 4 NFEs per edit. Our method is not only fast, but also significantly outperforms state-of-the-art multi-step diffusion editing techniques.
EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editing
The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of California, Los Angeles(加州大学洛杉矶分校);Microsoft AI Superintelligence(微软人工智能超智实验室);Lambda, Inc(Lambda公司)
DIRECT: Video Mashup Creation via Hierarchical Multi-Agent Planning and Intent-Guided Editing
School of Electronics Engineering and Computer Science, Peking University(北京大学电子工程与计算机科学学院);School of Computer Science, Peking University(北京大学计算机科学学院);Huazhong University of Science and Technology(华中科技大学)
The CLIP Model is Secretly an Image-to-Prompt Converter
Xidian University(西安电子科技大学);Northwestern Polytechnical University(西北工业大学);The University of Adelaide(阿德莱德大学);Australian Institute for Machine Learning(澳大利亚机器学习研究院)
NVIDIA(NVIDIA公司);University of Toronto(多伦多大学);Vector Institute(向量研究所);Harvard University(哈佛大学);Kempner Institute(凯姆纳研究所);IAIFI(IAIFI机构);Microsoft Research(微软研究院);Carnegie Mellon University(卡内基梅隆大学);Courant Institute, New York University(纽约大学应用数学学院);ML Lab at Capital Fund Management (CFM)(Capital Fund Management (CFM)机器学习实验室)
IReNe: Instant Recoloring of Neural Radiance Fields
Arquimea Research Center(阿克梅亚研究中心);Universitat Politècnica de Catalunya(加泰罗尼亚理工大学);Universidad Rey Juan Carlos(胡安·卡洛斯国王大学);Volinga AI(Volinga AI公司);SEDDI(SEDDI机构);Institut de Robòtica i Informàtica Industrial, CSIC-UPC(西班牙国家研究委员会-加泰罗尼亚理工大学机器人与工业信息研究所);Universidad de las Palmas de Gran Canaria(大加那利岛拉斯帕尔马斯大学);IUSIANI(IUSIANI机构);Universidad Politécnica de Madrid(马德里理工大学)
IDOL: Instant Photorealistic 3D Human Creation from a Single Image
Nanjing University(南京大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Tsinghua University(清华大学);Tencent(腾讯);Shenzhen University of Advanced Technology(深圳理工大学)