衡量通过细粒度人格提示生成的合成数据的词汇多样性
Measuring Lexical Diversity of Synthetic Data Generated through Fine-Grained Persona Prompting
AI总结：
本文通过词汇多样性指标评估人格提示生成的合成数据，发现合成提示多样性低于人工提示，人格提示可提升多样性但细粒度细节增益有限。
摘要：
细粒度人格（fine-grained personas）近期已被用于生成“多样化”的合成数据，以用于大型语言模型（LLMs）的预训练和监督微调。在本工作中，我们使用一套词汇多样性与冗余度指标来衡量由人格驱动的合成生成的提示与响应的多样性。首先，我们发现合成提示/指令的多样性显著低于人类撰写的提示/指令。接着，我们从不同规模的LLMs中采样响应，并分别使用细粒度和粗粒度的人格描述，以探究人格描述中的细粒度细节在多大程度上有助于生成文本的多样性。我们的结果表明，与不使用人格的提示相比，人格提示能产生更高的词汇多样性，尤其是在较大的模型中。相比之下，与仅在提示中指定长度截断相比，添加细粒度人格细节所带来的多样性增益微乎其微。
英文摘要：
Fine-grained personas have recently been used for generating 'diverse' synthetic data for pre-training and supervised fine-tuning of Large Language Models (LLMs). In this work, we measure the diversity of persona-driven synthetically generated prompts and responses with a suite of lexical diversity and redundancy metrics. First, we find that synthetic prompts/instructions are significantly less diverse than human-written ones. Next, we sample responses from LLMs of different sizes with fine-grained and coarse persona descriptions to investigate how much fine-grained detail in persona descriptions contribute to generated text diversity. Our results indicate that persona prompting produces higher lexical diversity than prompting without personas, particularly in larger models. In contrast, adding fine-grained persona details yields minimal gains in diversity compared to simply specifying a length cutoff in the prompt.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
