arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.04513cs.CLcs.AIcs.LG
收藏

可切换决策:动态神经生成网络

Switchable Decision: Dynamic Neural Generation Networks

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Shujian Zhang, Korawat Tanwisuth, Chengyue Gong, Pengcheng He, Mingyuan Zhou

AI总结:

针对自回归生成模型推理慢的问题,提出动态神经生成网络与可切换决策,按实例分配计算并约束优化质量与成本,在多类NLP任务中降本且不损失准确率。

摘要:

自回归生成模型在摘要、问答和分类等多种NLP任务上取得了具有竞争力的性能。然而,它们也因推理速度缓慢而闻名,这使其难以部署到实时应用中。我们提出一种可切换决策,通过为每个数据实例动态分配计算资源来加速推理。我们的动态神经生成网络能够自动决定在何处跳过计算,以及如何通过约束优化平衡质量与计算成本,从而强制采用高效推理路径并确定最优折中方案。在问答、摘要和分类基准上的实验表明,我们的方法在推理期间受益于更低的计算成本,同时保持相同的准确率。大量实验和消融研究证明,我们的方法具有通用性、有效性,并能为许多NLP任务带来收益。

英文摘要:

Auto-regressive generation models achieve competitive performance across many different NLP tasks such as summarization, question answering, and classifications. However, they are also known for being slow in inference, which makes them challenging to deploy in real-time applications. We propose a switchable decision to accelerate inference by dynamically assigning computation resources for each data instance. Automatically making decisions on where to skip and how to balance quality and computation cost with constrained optimization, our dynamic neural generation networks enforce the efficient inference path and determine the optimized trade-off. Experiments across question answering, summarization, and classification benchmarks show that our method benefits from less computation cost during inference while keeping the same accuracy. Extensive experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑