我应该选择哪种型号？文本分类任务的成本/质量权衡
Which Model Shall I Choose? Cost/Quality Trade-offs for Text Classification Tasks
摘要：
行业从业者总是面临着在不同考虑因素下选择合适的部署模型的问题，例如最大限度地提高对生产至关重要的指标，或者考虑到财务问题降低总成本。在这项工作中，我们专注于文本分类任务，并提出了定量分析这一挑战。使用分类精度作为主要指标，我们评估了分类器在各种模型（包括大语言模型）中的性能，以及它们的相关成本，包括注释成本、训练（微调）成本和推理成本。然后，我们讨论了模型选择的情况下，如有大量的样本需要进行推理。我们希望我们的工作将帮助人们更好地理解文本分类任务的成本/质量权衡。
英文摘要：
Industry practitioners always face the problem of choosing the appropriate model for deployment under different considerations, such as to maximize a metric that is crucial for production, or to reduce the total cost given financial concerns. In this work, we focus on the text classification task and present a quantitative analysis for this challenge. Using classification accuracy as the main metric, we evaluate the classifiers' performances for a variety of models, including large language models, along with their associated costs, including the annotation cost, training (fine-tuning) cost, and inference cost. We then discuss the model choices for situations like having a large number of samples needed for inference. We hope our work will help people better understand the cost/quality trade-offs for the text classification task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
