UIPress:将光学令牌压缩引入UI到代码生成
UIPress: Bringing Optical Token Compression to UI-to-Code Generation
AI总结:
本文提出UIPress,一种轻量级学习压缩模块,通过结合深度可分离卷积、元素引导的空间重加权和Transformer细化,将UI截图的视觉令牌从约6700压缩到256,结合LoRA实现表示差距桥接,提升效率并优于现有方法。
摘要:
UI到代码生成需要视觉语言模型(VLMs)从单张截图生成数千个结构化HTML/CSS令牌,使视觉令牌效率至关重要。现有压缩方法要么在推理时使用任务无关的启发式选择令牌,要么零出低注意力特征而不实际缩短序列,既不能真正减少prefill延迟,也无法适应UI截图非均匀的信息密度。同时,光学(编码器侧学习)压缩在文档OCR中表现出色,但此前无工作将其适应于UI到代码生成。我们提出UIPress,一种轻量级学习压缩模块,插入到冻结的ViT编码器和LLM解码器之间。UIPress结合深度可分离卷积、元素引导的空间重加权和Transformer细化,将约6700个视觉令牌压缩到256的固定预算。结合解码器上的低秩适应(LoRA)以桥接表示差距,整个系统仅增加约21.7M可训练参数(8B基础模型的0.26%)。在Design2Code上与四个基线的公平比较中,UIPress在256个令牌时达到CLIP分数0.8127,优于未压缩基线+7.5%和最强的推理时方法+4.6%,同时实现9.1倍的时间到第一个令牌加速。据我们所知,UIPress是首个针对UI到代码任务的编码器侧学习压缩方法。
英文摘要:
UI-to-Code generation requires vision-language models (VLMs) to produce thousands of tokens of structured HTML/CSS from a single screenshot, making visual token efficiency critical. Existing compression methods either select tokens at inference time using task-agnostic heuristics, or zero out low-attention features without actually shortening the sequence -- neither truly reduces prefill latency or adapts to the non-uniform information density of UI screenshots. Meanwhile, optical (encoder-side learned) compression has shown strong results for document OCR, yet no prior work has adapted this paradigm to UI-to-Code generation. We propose UIPress, a lightweight learned compression module inserted between the frozen ViT encoder and the LLM decoder of Qwen3-VL-8B. UIPress combines depthwise-separable convolutions, element-guided spatial reweighting, and Transformer refinement to compress ${\sim}$6{,}700 visual tokens to a fixed budget of 256. Together with Low-Rank Adaptation (LoRA) on the decoder to bridge the representation gap, the entire system adds only ${\sim}$21.7M trainable parameters (0.26\% of the 8B base model). Under a fair comparison on the same base model against four baselines on Design2Code, UIPress at 256 tokens achieves a CLIP score of 0.8127, outperforming the uncompressed baseline by +7.5\% and the strongest inference-time method by +4.6\%, while delivering 9.1$\times$ time-to-first-token speedup. To the best of our knowledge, UIPress is the first encoder-side learned compression method for the UI-to-Code task.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。