When2Speak: 一个多党对话中大型语言模型的时间参与和发言时机的数据集
When2Speak: A Dataset for Temporal Participation and Turn-Taking in Multi-Party Conversations for Large Language Models
AI总结：
本文提出When2Speak数据集，通过四阶段生成流程学习群体互动中的发言时机，通过强化学习优化模型，提升多党对话中的发言准确性与自然度。
摘要：
大型语言模型（LLMs）在生成上下文合适响应方面表现出色，但在多党对话中决定何时发言同样关键。我们引入When2Speak，一个基于现实的合成数据集和四阶段生成流程，用于学习群体互动中的干预时机。数据集包含超过215,000个示例，源自16,000次包含2-6名发言者的对话，涵盖多样化的对话风格、语气和参与者动态，并在每个回合显式建模SPEAK vs. SILENT决策。我们的流程结合了现实世界的基础、结构化增强、受控转录合成和可微调的监督，并完全开源以支持可重复性和适应特定领域的对话规范。在多个模型家族中，对When2Speak的监督微调（SFT）显著优于零样本基线（例如，平均宏F1增加超过4B+参数模型的60%，最大增加为120%）。然而，SFT训练的模型仍系统性地过于保守，错过近一半的必要干预，如通过遗漏干预率（MIR）显示，平均为0.50，即使在较大模型规模下也存在。为了解决这一限制，我们应用了具有不对称奖励塑造的强化学习，将MIR降低到0.186-0.218，并将召回率从0.479提高到0.78-0.81。我们的发现表明，时间参与是对话智能的一个独特且可训练的维度，而基于现实的合成数据为使LLM更自然和适当地参与多党互动提供了有效且可扩展的途径。
英文摘要：
Large Language Models (LLMs) excel at generating contextually appropriate responses but remain poorly calibrated for multi-party conversations, where deciding when to speak is as critical as what to say. In such settings, naively responding at every turn leads to excessive interruptions and degraded conversational coherence. We introduce When2Speak, a grounded synthetic dataset and four-stage generation pipeline for learning intervention timing in group interactions. The dataset comprises over 215,000 examples derived from 16,000 conversations involving 2-6 speakers, spanning diverse conversational styles, tones, and participant dynamics, and explicitly modeling SPEAK vs. SILENT decisions at each turn. Our pipeline combines real-world grounding, structured augmentation, controlled transcript synthesis, and fine-tuning-ready supervision, and is fully open-sourced to support reproducibility and adaptation to domain-specific conversational norms. Across multiple model families, supervised fine-tuning (SFT) on When2Speak significantly outperforms zero-shot baselines (e.g., the average Macro F1 increase across 4B+ parameter models was 60%, with the largest increase being 120%). However, SFT-trained models remain systematically over-conservative, missing nearly half of warranted interventions as seen through the Missed Intervention Rate (MIR), which was on average 0.50 and is noticed even at larger model sizes. To address this limitation, we apply reinforcement learning with asymmetric reward shaping, which reduces MIR to 0.186-0.218 and increases recall from 0.479 to 0.78-0.81. Our findings establish that temporal participation is a distinct and trainable dimension of conversational intelligence, and that grounded synthetic data provides an effective and scalable pathway for enabling LLMs to participate more naturally and appropriately in multi-party interactions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
