Hydra：在单一视觉-语言模型中统一文档检索与生成
Hydra: Unifying Document Retrieval and Generation in a Single Vision-Language Model
AI总结：
Hydra通过单一视觉-语言模型实现文档检索与生成的统一，减少内存和系统复杂性，通过LoRA适配器在推理时切换，提升效率并保持生成质量。
摘要：
视觉文档理解通常需要单独的检索和生成模型，这会增加内存和系统复杂性。我们提出了Hydra，一种双头方法，提供ColBERT风格的晚交互检索和自回归生成，从单一视觉-语言模型中实现。一个训练仅用于检索的LoRA适配器在推理时可切换：启用时产生多向量嵌入；禁用时恢复基础模型的生成质量，426个语言模型权重张量与重新加载的Qwen3.5-4B完全相同。我们识别出两种可能破坏生成的故障模式（注意力模式恢复和lm_head保留）以及一个效率要求（KV缓存意识解码）；Hydra通过结构方式规避前两种，并在解码循环中处理第三种。我们发布了两种规模的Hydra-4B和Hydra-0.8B，共享LoRA超参数（r=32，alpha=32）和优化配方；数据混合和投影维度在不同规模中有所不同。单一模型设计将4B峰值GPU内存从28.85GB减少到10.77GB（62.7%的减少），在0.8B规模下从5.79GB减少到2.37GB（59.1%的减少），相对于共居的双模型部署。受控的消融实验发现GritLM风格的联合训练在评估模式中与Hydra的检索-only训练相当，而其LoRA-on生成模式崩溃。在Qwen2.5-Omni-3B上的概念验证保持了在非Qwen3.5背骨上的生成等价性，并在2-8 pp范围内转移图像检索，通过冻结的Whisper编码器实现了零样本音频检索。
英文摘要：
Visual document understanding typically requires separate retrieval and generation models, doubling memory and system complexity. We present Hydra, a dual-head approach that provides both ColBERT-style late-interaction retrieval and autoregressive generation from a single vision-language model. A single LoRA adapter, trained only for retrieval, is toggled at inference: enabling it produces multi-vector embeddings; disabling it recovers the base model's generation quality, with 426 of 426 language-model weight tensors byte-for-byte identical to a freshly-loaded Qwen3.5-4B. We identify two failure modes that can silently break generation in retrieval-fine-tuned VLMs (attention-mode restoration and lm_head preservation) plus an efficiency requirement (KV-cache-aware decoding); Hydra sidesteps the first two structurally and addresses the third in the decode loop. We release two scales, Hydra-4B and Hydra-0.8B, sharing LoRA hyperparameters (r=32, alpha=32) and optimisation recipe; data mix and projection dim differ across scales. The single-model design cuts peak GPU memory from 28.85 GB to 10.77 GB at 4B (62.7% reduction) and from 5.79 GB to 2.37 GB at 0.8B (59.1%) relative to a co-resident two-model deployment. A controlled ablation finds GritLM-style joint training matches Hydra's retrieval-only training on the evaluated modes while its LoRA-on generation mode collapses. A proof-of-concept on Qwen2.5-Omni-3B preserves generation equivalence on a non-Qwen3.5 backbone and transfers image retrieval within 2-8 pp of Hydra-4B, with zero-shot audio retrieval emerging through the frozen Whisper encoder.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
