arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

TAP-Path:面向高效且可靠的病理学基础模型的任务自适应结构与Token剪枝

TAP-Path: Task-Adaptive Structural and Token Pruning for Efficient and Trustworthy Pathology Foundation Models

Mehedi Hasan, Ashfak Yeafi, Md Khairul Islam

arXiv 2609.04071首次发表:更新:

发表机构

Brac University; Khulna University of Engineering & Technology; Hobart and William Smith Colleges(BRAC大学; 库尔纳工程技术大学; 霍巴特与威廉史密斯学院)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

本文提出TAP-Path框架,通过重构Virchow2编码器实现任务自适应压缩,在减少参数与计算量的同时,提升了病理学基础模型的准确率-效率权衡及内外评估可靠性。

AI 中文摘要

病理学基础模型可改进组织病理学的可迁移表示学习,但近期的性能提升往往依赖于具有数亿参数的编码器,推理成本较高。本文提出TAP-Path,这是一种任务自适应压缩框架,它直接重构预训练的Virchow2编码器,而非将其蒸馏为单独的学生模型。TAP-Path结合了验证驱动的Transformer块选择、冗余块的物理移除、输入自适应的Patch Token剪枝、多深度特征恢复以及轻量级门控任务头。最终模型在剪枝后保留了32个Transformer块中的24个和70%的Patch Token,编码器参数减少24.96%(从631.24M降至473.70M),分析编码器计算量减少35.20%(从340.13G FLOPs降至220.40G FLOPs)。在3个任务头优化种子下,TAP-Path在32类组织病理学基准上取得了87.98±0.067%的测试准确率、81.26±0.49%的平衡准确率和82.38±0.48%的宏F1值,而完整Virchow2的对应值为86.89%,UNI2-h为87.67%。TAP-Path的Brier分数为0.1800±0.0005,故障检测AUROC为0.9047±0.0060。仅验证的稀有感知目标在二次操作分析中提升了稀有类别的平衡准确率。对433个CPTAC样本的冻结外部评估取得了91.22±0.83%的准确率和91.10±0.81%的平衡准确率。这些结果表明,任务自适应的结构与Token稀疏化可改善大型病理学基础模型的准确率-效率权衡,同时在内部和外部评估下保持可靠性。

英文摘要

Pathology foundation models improve transferable representation learning for histopathology, but recent gains often rely on encoders with hundreds of millions of parameters and high inference cost. We propose TAP-Path, a task-adaptive compression framework that directly restructures a pretrained Virchow2 encoder rather than distilling it into a separate student. TAP-Path combines validation-driven transformer-block selection, physical removal of redundant blocks, input-adaptive patch-token pruning, multi-depth feature recovery, and a lightweight gated task head. The final model retains 24 of 32 transformer blocks and 70% of patch tokens after pruning, reducing encoder parameters by 24.96% (631.24M to 473.70M) and analytical encoder compute by 35.20% (340.13G to 220.40G FLOPs). Across three task-head optimization seeds, TAP-Path achieved $87.98 \pm 0.067%$ test accuracy, $81.26 \pm 0.49%$ balanced accuracy, and $82.38 \pm 0.48%$ macro-F1 on a 32-class histopathology benchmark, compared with 86.89% for full Virchow2 and 87.67% for UNI2-h. TAP-Path achieved a Brier score of $0.1800 \pm 0.0005$ and failure-detection AUROC of $0.9047 \pm 0.0060$. A validation-only rare-aware objective improved rare-class balanced accuracy in a secondary operating analysis. Frozen external evaluation on 433 CPTAC samples yielded $91.22 \pm 0.83%$ accuracy and $91.10 \pm 0.81%$ balanced accuracy. These results show that task-adaptive structural and token sparsification can improve the accuracy-efficiency trade-off of large pathology foundation models while preserving reliability under internal and external evaluation.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑