arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.07682cs.CL
收藏

TRIM:用于高效语言生成的标记减少与推理建模

TRIM: Token Reduction and Inference Modeling for Cost-Effective Language Generation

AI Research, JPMorganChase(人工智能研究,摩根大通)

Alfredo Garrachón Ruiz, Tomás de la Rosa, Daniel Borrajo

AI总结:

TRIM通过省略冗余词和重建答案,实现高效语言生成,减少计算成本并保持准确性。

摘要:

大型语言模型(LLMs)的高推理成本带来了挑战,尤其是对于需要长输出的任务。然而,自然语言往往包含冗余,这为优化提供了机会。我们观察到,当适当提示时,LLMs可以生成浓缩的语言(即保留核心意义的简洁输出)。我们提出TRIM,是一种节省计算成本的流水线,在推理过程中,LLM会根据上下文省略一组语义不相关且容易推断的词。然后,经过专门训练的较小语言模型,以较低的推理成本将浓缩的答案重建为理想答案。我们的实验显示了有希望的结果,特别是在提出的NaLDA评估数据集上,专注于重建任务,平均在GPT-4o上节省了19.4%的标记,且评估指标仅略有下降。这表明,该方法可以在语言处理任务中有效平衡效率和准确性。

英文摘要:

The high inference cost of Large Language Models (LLMs) poses challenges, especially for tasks requiring lengthy outputs. However, natural language often contains redundancy, which presents an opportunity for optimization. We have observed that LLMs can generate distilled language (i.e., concise outputs that retain essential meaning) when prompted appropriately. We propose TRIM, a pipeline for saving computational cost in which the LLM omits a predefined set of semantically irrelevant and easily inferable words based on the context during inference. Then, a specifically trained smaller language model with lower inference cost reconstructs the distilled answer into the ideal answer. Our experiments show promising results, particularly on the proposed NaLDA evaluation dataset focused on the reconstruction task, with 19.4% saved tokens on average for GPT-4o and only a tiny decrease in evaluation metrics. This suggests that the approach can effectively balance efficiency and accuracy in language processing tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑