发表机构
Korea University（高丽大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对现有提示微调结构的局限，提出HiVe框架，通过训练构建提示层次结构、推理时采用V-MoE机制实现输入依赖的提示专业化，在多任务中性能优于基线方法。
AI 中文摘要
随着大语言模型（LLMs）规模不断扩大，参数高效微调（PEFT）已成为全参数适配的实用替代方案。提示微调虽有效，但现有方法要么采用扁平提示结构，要么采用固定提示组合的分层结构，限制了提示的自适应专业化。为解决这一局限，我们提出HiVe，一种对提示进行多级别建模、支持输入依赖专业化的提示微调框架。HiVe在训练时利用任务间关系构建提示层次结构，并在推理时采用垂直混合专家（V-MoE）机制，为每个输入组合出所需专业化级别的提示。实验表明，在各类任务中，HiVe始终优于强大的提示微调基线方法。
英文摘要
As large language models (LLMs) continue to scale, parameter-efficient fine-tuning (PEFT) has become a practical alternative to full-parameter adaptation. Prompt tuning is effective, but existing approaches either use flat prompt structures or hierarchical structures with fixed prompt composition, limiting adaptive prompt specialization. To address this limitation, we propose HiVe, a prompt tuning framework that models prompts at multiple levels and enables input-dependent specialization. HiVe constructs a prompt hierarchy by leveraging inter-task relationships during training, and employs a vertical mixture-of-experts (V-MoE) mechanism at inference time to compose prompts up to the level of specialization required for each input. Experiments show that HiVe consistently outperforms strong prompt tuning baselines across diverse tasks.
CommentsAccepted to the EMNLP 2026 Main Conference
