文本引导的图像编辑技术允许用户通过自然语言指令变换和合成图像，具备极强的灵活性。然而，现有大多数图像编辑模型会盲目尝试遵循所有用户指令，即便这些指令本身不可行或相互矛盾，往往会生成不合逻辑的输出结果。为解决上述挑战，我们提出一种上下文感知的图像编辑方法，名为CAMILA（Context-Aware Masking for Image Editing with Language Alignment，面向语言对齐的图像编辑上下文感知掩码方法）。CAMILA用于验证指令与图像之间的上下文一致性，确保仅对指定区域执行相关编辑，同时忽略不可执行的指令。为全面评估该新方法，我们构建了单指令和多指令图像编辑数据集，数据集中包含不可行请求。我们的方法相比当前最优模型取得了更优的性能和更高的语义对齐度，证明其在保留图像完整性的同时，能有效应对复杂指令带来的挑战。
英文摘要：
Text-guided image editing has been allowing users to transform and synthesize images through natural language instructions, offering considerable flexibility. However, most existing image editing models naively attempt to follow all user instructions, even if those instructions are inherently infeasible or contradictory, often resulting in nonsensical output. To address these challenges, we propose a context-aware method for image editing named as CAMILA (Context-Aware Masking for Image Editing with Language Alignment). CAMILA is designed to validate the contextual coherence between instructions and the image, ensuring that only relevant edits are applied to the designated regions while ignoring non-executable instructions. For comprehensive evaluation of this new method, we constructed datasets for both single- and multi-instruction image editing, incorporating the presence of infeasible requests. Our method achieves better performance and higher semantic alignment than state-of-the-art models, demonstrating its effectiveness in handling complex instruction challenges while preserving image integrity.
Google Research（谷歌研究院）；The Hebrew University of Jerusalem（希伯来大学）
Text-driven image and video diffusion models have recently achieved unprecedented generation realism. While diffusion models have been successfully applied for image editing, very few works have done so for video editing. We present the first diffusion-based method that is able to perform text-based motion and appearance editing of general videos. Our approach uses a video diffusion model to combine, at inference time, the low-resolution spatio-temporal information from the original video with new, high resolution information that it synthesized to align with the guiding text prompt. As obtain…
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models
Rutgers University（新泽西罗格斯大学）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；Red Hat AI Innovation（红帽AI创新）；Google DeepMind（谷歌DeepMind）；NYU（纽约大学）；Walmart Global Tech（沃尔玛全球技术）；NEC Labs America（NEC美国实验室）；Massachusetts Institute of Technology（麻省理工学院）；ANU（澳大利亚国立大学）；UT Arlington（德克萨斯大学阿灵顿分校）
DaSCI research institute（DaSCI研究所）；University of Granada（格拉纳达大学）；Télécom-Paris（巴黎电信学院）
In this work, we propose a novel architecture for face age editing that can produce structural modifications while maintaining relevant details present in the original image. We disentangle the style and content of the input image and propose a new decoder network that adopts a style-based strategy to combine the style and content representations of the input image while conditioning the output on the target age. We go beyond existing aging methods allowing users to adjust the degree of structure preservation in the input image during inference. To this purpose, we introduce a masking mechanis…
HTS-Attack: Heuristic Token Search for Jailbreaking Text-to-Image Models
Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；CFAR；IHPC；Mohammed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；University of Oxford（牛津大学）
Hierarchical Conditioning of Diffusion Models Using Tree-of-Life for Studying Species Evolution
Virginia Tech（弗吉尼亚理工大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；University of California, Irvine（加利福尼亚大学欧文分校）；Tulane University（杜兰大学）；Battelle（巴特尔公司）；Duke University（杜克大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；The Ohio State University（俄亥俄州立大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany（马克斯·普朗克研究所信息学院，萨尔兰信息学院，德国萨尔布吕肯）；Department of Language Science and Technology, Saarland University, Saarbrücken, Germany（语言科学与技术系，萨尔兰大学，德国萨尔布吕肯）
