Tiny Titans：较小的大语言模型能否在现实世界的会议摘要生成任务中超常发挥？
Tiny Titans: Can Smaller Large Language Models Punch Above Their Weight in the Real World for Meeting Summarization?
AI总结：
针对大语言模型现实部署成本高的问题，本文对比微调后小型LLMs与零样本大型LLMs的会议摘要性能，发现7.8亿参数的FLAN-T5可媲美甚至超越70亿到700亿+参数的大模型，为工业部署提供高性价比方案。
摘要：
大语言模型（LLMs）已展现出无需在任务特定数据集上显式微调即可解决广泛任务的出色能力。然而，在现实世界中部署LLMs并非易事，因为这需要大量计算资源。本文中，我们研究了更小型、紧凑的LLMs是否能作为相对更大的LLMs的良好替代方案，以解决现实世界中使用LLMs相关的高昂成本问题。在此背景下，我们在真实工业环境中研究了会议摘要生成任务，并开展了大量实验，将经过微调的紧凑LLMs（如FLAN-T5、TinyLLaMA、LiteLLaMA）与零样本更大LLMs（如LLaMA-2、GPT-3.5、PaLM-2）的性能进行了对比。我们观察到，大多数较小的LLMs即使经过微调，在会议摘要数据集上也未能超越零样本的更大LLMs。但一个值得注意的例外是拥有7.8亿参数的FLAN-T5，其性能与许多零样本更大LLMs（参数规模从70亿到700亿以上）相当甚至更优，同时体积却小得多。这使得FLAN-T5这类紧凑LLMs成为现实世界工业部署中合适的高性价比解决方案。
英文摘要：
Large Language Models (LLMs) have demonstrated impressive capabilities to solve a wide range of tasks without being explicitly fine-tuned on task-specific datasets. However, deploying LLMs in the real world is not trivial, as it requires substantial computing resources. In this paper, we investigate whether smaller, compact LLMs are a good alternative to the comparatively Larger LLMs2 to address significant costs associated with utilizing LLMs in the real world. In this regard, we study the meeting summarization task in a real-world industrial environment and conduct extensive experiments by comparing the performance of fine-tuned compact LLMs (e.g., FLAN-T5, TinyLLaMA, LiteLLaMA) with zero-shot larger LLMs (e.g., LLaMA-2, GPT-3.5, PaLM-2). We observe that most smaller LLMs, even after fine-tuning, fail to outperform larger zero-shot LLMs in meeting summarization datasets. However, a notable exception is FLAN-T5 (780M parameters), which performs on par or even better than many zero-shot Larger LLMs (from 7B to above 70B parameters), while being significantly smaller. This makes compact LLMs like FLAN-T5 a suitable cost-efficient solution for real-world industrial deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
