用于可控数据生成的软提示混合
Mixture of Soft Prompts for Controllable Data Generation
AI总结：
提出软提示混合（MSP）方法，将LLM作为数据增强工具而非直接预测工具，通过参数高效的可控生成和去噪机制提升数据质量，在三个基准测试中取得最先进结果。
摘要：
当目标输出遵循自然语言模式时，大型语言模型（LLMs）能有效地生成流畅的文本。然而，结构化预测任务将输出格式限制在有限的本体论中，导致即使是极大的模型也会遇到困难，因为它们在训练时从未考虑过此类限制。在由于领域转移和资源限制而经常出现的少样本学习场景中，使用LLMs进行直接预测的困难进一步加剧。我们彻底改变思路，将LLM作为数据增强工具而非直接预测工具。我们提出的软提示混合（Mixture of Soft Prompts, MSP）是一种以可控方式生成数据的参数高效方法。进一步应用去噪机制来提高合成数据的质量。自动指标表明，我们的方法能够生成多样且自然的文本，同时保留标签语义。此外，与强基线相比，MSP在三个基准测试中取得了最先进的结果。我们的方法为将LLM应用于复杂预测任务提供了一种替代的以数据为中心的方法。
英文摘要：
Large language models (LLMs) effectively generate fluent text when the target output follows natural language patterns. However, structured prediction tasks confine the output format to a limited ontology, causing even very large models to struggle since they were never trained with such restrictions in mind. The difficulty of using LLMs for direct prediction is exacerbated in few-shot learning scenarios, which commonly arise due to domain shift and resource limitations. We flip the problem on its head by leveraging the LLM as a tool for data augmentation rather than direct prediction. Our proposed Mixture of Soft Prompts (MSP) serves as a parameter-efficient procedure for generating data in a controlled manner. Denoising mechanisms are further applied to improve the quality of synthesized data. Automatic metrics show our method is capable of producing diverse and natural text, while preserving label semantics. Moreover, MSP achieves state-of-the-art results on three benchmarks when compared against strong baselines. Our method offers an alternate data-centric approach for applying LLMs to complex prediction tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
