ORCA：面向无需训练的3D CT视觉令牌压缩的器官质心聚合方法
ORCA: ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression
浏览论文内容
中文总结 AI 辅助
针对3D CT视觉令牌压缩的痛点，提出无需训练的即插即用ORCA方法，在多数据集、多任务上实现显著压缩比与速度提升，性能优于现有方法。
中文摘要 AI 辅助
输入视觉语言模型的3D CT扫描会生成一长串视觉令牌，每个体积通常有数千到数万个，该序列必须经过压缩才能被语言模型处理。令牌压缩在通用视觉领域已得到充分研究，但针对3D CT的研究很少。常见的基线方法是网格平均，它会对规则网格单元进行池化，可能将不同的解剖结构、病变和空气混合成一个令牌。我们提出了ORCA（器官质心聚合，ORgan-Centroid Aggregation），一种用于3D CT的令牌压缩器，它以器官为指导合并相邻令牌，并添加每个区域质心的正弦编码以保留空间布局，从而保留下游模型所需的解剖信息。ORCA无需训练且即插即用，可生成可调的令牌集，无需任何模型更改或文本查询。我们在两个数据集（CT-RATE和Merlin）及五个编码器上对其进行评估，评估涵盖两类任务：五个类别（大小、密度、位置、纹理和疾病）的属性预测，以及文本生成（视觉问答和报告生成）。在相同令牌预算下，ORCA始终优于现有压缩方法，它将视觉上下文压缩64倍，KV缓存压缩50倍，处理每个体积的速度提升31倍。代码已发布在此https URL。
英文摘要
A 3D CT scan entering a vision-language model produces a long sequence of visual tokens, often thousands to tens of thousands per volume, and this sequence must be compressed before a language model can consume it. Token compression is well studied in general vision, but little of it targets 3D CT specifically. A common baseline is grid average, which pools regular grid cells and can blend distinct anatomy, lesion, and air into one token. We present \textbf{ORCA} (ORgan-Centroid Aggregation), a token compressor for 3D CT. It merges adjacent tokens with organ guidance and adds a sinusoidal encoding of each region's centroid to preserve spatial layout. This preserves the anatomical information a downstream model needs. ORCA is training-free and plug-and-play, producing an adjustable token set without any model change or text query. We evaluate it across two datasets (CT-RATE and Merlin) and five encoders. The evaluation spans two task types: attribute prediction over five families (size, density, location, texture, and disease) and text generation (visual question answering and report generation). At matched token budgets, ORCA improves consistently over existing compression methods. It shrinks the visual context $64\times$ and its KV-cache $50\times$, and is $31\times$ faster to process each volume. Code released at https://github.com/renjie-liang/ORCA-3DCT.
