arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

条件秩分配用于分类学感知的医学语言模型适配

Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation

Guangyuan Dong, Ziwei Hong, Xuehao Zhou, Zidong Yu, Bingchen Liu, Kehan Liu, Chuang Liu, Rong Fu, Yuchao Hou

arXiv 2610.06765首次发表:更新:

发表机构

National University of Singapore; Shanghai General Hospital; Shanghai Jiao Tong University School of Medicine; University of Pennsylvania; Shandong University; Syracuse University(新加坡国立大学; 上海市第一人民医院; 上海交通大学医学院; 宾夕法尼亚大学; 山东大学; 雪城大学)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

提出ARBOR,一种基于条件秩分配的参数高效方法,通过共享低秩基和门控机制实现医学问答的专科适配,在多个基准上优于LoRA和MoELoRA。

AI 中文摘要

医学问答涉及不同的专业和临床操作,这些可能受益于不同的适配方向。我们提出ARBOR,一种参数高效的方法,为每个问题从共享的低秩基中选择秩一分量。一个加性门结合了问题表示、专业标签、操作标签及其交互;一个学习到的系数缩放适配器残差。在正交、等概率子任务下的说明性分离表明,条件选择可以避免固定更新在相同活跃秩下所面临的近似下限。这一结果激励了设计,但并未声称对医学语料库有相应的界。在Qwen3-8B上,跨CMB、CMExam、MedQA和MedMCQA,五次种子实验在基准上平均准确率为69.69%,分别超过LoRA r16和MoELoRA 1.26和1.30个百分点。报告的优势相对于LoRA r16,随着训练从1个专业扩展到7个专业,从0.08增加到1.94个百分点。标签扰动和原子掩码支持临床路由的有用性,而原子簇与提供的专业标签对齐(调整兰德指数0.62)。校准、迁移和测量成本进一步表征了该方法。这些发现支持医学问答的结构化条件适配,但临床安全性和更广泛的部署尚未测试。

英文摘要

Medical question answering spans specialties and clinical operations that may benefit from different adaptation directions. We propose ARBOR, a parameter-efficient method that selects rank-one components from a shared low-rank basis for each question. An additive gate combines question representations, specialty tags, operation tags, and their interaction; a learned coefficient scales the adapter residual. An illustrative separation under orthogonal, equiprobable subtasks shows how conditional selection can avoid an approximation floor faced by a fixed update with the same active rank. This result motivates the design without asserting a corresponding bound for medical corpora. On Qwen3-8B across CMB, CMExam, MedQA, and MedMCQA, five-seed experiments yield 69.69% mean accuracy across benchmarks, exceeding LoRA r16 and MoELoRA by 1.26 and 1.30 percentage points, respectively. The reported advantage over LoRA r16 increases from 0.08 to 1.94 points as training expands from one to seven specialties. Tag perturbations and atom masking support the usefulness of clinical routing, while atom clusters align with the supplied specialty labels (adjusted Rand index 0.62). Calibration, transfer, and measured costs further characterize the method. These findings support structured conditional adaptation for medical QA, while leaving clinical safety and broader deployment untested.

CommentsAccepted at IEEE BIBM 2026

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑