迈向面向所有人的敏捷文本分类器
Towards Agile Text Classifiers for Everyone
AI总结：
本文提出敏捷文本分类方法，通过小规模目标数据集对大语言模型进行提示微调，仅需80个示例即可达到SOTA性能，实现安全分类器的快速定制与迭代。
摘要：
基于文本的安全分类器被广泛用于内容审核，并日益用于调整生成式语言模型的行为——这是数字助手和聊天机器人安全方面日益受关注的话题。然而，不同的政策需要不同的分类器，且安全政策本身也通过迭代和适应来改进。本文介绍并评估了敏捷文本分类的方法，即使用小型、有针对性的数据集来训练分类器，这些数据集可以针对特定政策快速开发。通过对来自三个安全相关领域的7个数据集（包含15种标注方案）进行实验，我们得出了关键发现：使用少至80个示例的标注数据集对PaLM 62B等大型语言模型进行提示微调，即可实现最先进的性能。我们认为这能促成文本分类的范式转变，特别是对于支持更安全在线话语的模型。无需花费数月或数年收集数百万示例来尝试创建通用安全分类器，分类器可以使用由个人或小型组织创建的小型数据集进行微调，针对特定用例量身定制，并在一天的时间跨度内进行迭代和适应。
英文摘要：
Text-based safety classifiers are widely used for content moderation and increasingly to tune generative language model behavior - a topic of growing concern for the safety of digital assistants and chatbots. However, different policies require different classifiers, and safety policies themselves improve from iteration and adaptation. This paper introduces and evaluates methods for agile text classification, whereby classifiers are trained using small, targeted datasets that can be quickly developed for a particular policy. Experimenting with 7 datasets from three safety-related domains, comprising 15 annotation schemes, led to our key finding: prompt-tuning large language models, like PaLM 62B, with a labeled dataset of as few as 80 examples can achieve state-of-the-art performance. We argue that this enables a paradigm shift for text classification, especially for models supporting safer online discourse. Instead of collecting millions of examples to attempt to create universal safety classifiers over months or years, classifiers could be tuned using small datasets, created by individuals or small organizations, tailored for specific use cases, and iterated on and adapted in the time-span of a day.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
