编辑比特,差异编码:面向视觉自回归模型的逐比特残差编辑
Edit the Bits, Diff the Codes: Bitwise Residual Editing for Visual Autoregressive Models
AI总结:
提出BitResEdit,一种无需训练的视觉自回归图像编辑方法,通过比特级源负引导和残差编码注入,在保持背景的同时实现强文本对齐。
摘要:
基于文本引导的图像编辑与视觉自回归(VAR)生成器需要控制模型采样的内容以及将采样变化写回图像代码的位置。现有的VAR编辑器主要操作于令牌流、特征或扁平的下一个令牌对数几率,忽略了逐比特残差VAR模型的两个原生结构:逐比特伯努利预测头和图像组装所用的加性多尺度残差代码域。我们提出BitResEdit,一种针对逐比特残差VAR生成器(如Infinity)的无训练编辑器。BitEdit通过沿共享编辑前缀上计算的源-目标对比倾斜后CFG的逐比特对数几率,执行源负引导,然后将每个更新投影到干净CFG采样器周围的闭式伯努利-KL信任域中。ResEdit将采样的比特转换为每尺度连续代码残差,用定位掩码对其进行门控,并通过生成器的原生尺度求和重新注入。它们共同将决策时的比特引导与组合时的代码组合耦合,使得被掩码的潜在特征通过代码算术精确保留,同时在目标区域内应用局部化的尺度感知编辑。在PIE-Bench上使用Infinity-2B,BitResEdit在相同骨干的VAR编辑器中实现了最强的文本对齐,在编辑区域上的CLIP比最强先前的编辑器提高了+1.07,同时背景保持与其相当。消融实验表明BitEdit和ResEdit在目标对齐和背景保持中发挥互补作用。
英文摘要:
Text-guided image editing with visual autoregressive (VAR) generators requires controlling both what the model samples and where the sampled change is written back into the image code. Existing VAR editors mainly operate on token streams, features, or flat next-token logits, leaving two native structures of bitwise-residual VAR models underused: the per-bit Bernoulli prediction head and the additive multi-scale residual code field from which the image is assembled. We propose BitResEdit, a training-free editor for bitwise-residual VAR generators such as Infinity. BitEdit performs source-negative guidance by tilting the post-CFG per-bit log-odds along a source--target contrast computed on a shared edited prefix, then projects each update into a closed-form Bernoulli-KL trust region around the clean CFG sampler. ResEdit converts the sampled bits into per-scale continuous-code residuals, gates them with a localization mask, and re-injects them through the generator's native sum-of-scales. Together they couple decision-time bit guidance with combination-time code composition, so masked-out latent features are preserved exactly by code arithmetic while localized, scale-aware edits are applied inside the target region. On PIE-Bench with Infinity-2B, BitResEdit attains the strongest text alignment among same-backbone VAR editors, improving CLIP on the edited region by +1.07 over the strongest prior editor while keeping background preservation competitive with it. Ablations show BitEdit and ResEdit play complementary roles in target alignment and background preservation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。