ThoughtTrace: 理解真实世界LLM交互中的用户想法
ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions
AI总结：
本文提出ThoughtTrace，首个大规模数据集，通过配对真实多轮人机对话与用户自述想法（提示原因和对助手回复的反应），揭示用户认知动态，并展示其在行为预测和个性化对齐中的价值。
摘要：
对话式AI现已服务数十亿用户，但现有数据集仅捕捉用户所说，而非所想。我们引入ThoughtTrace，首个大规模数据集，将真实世界多轮人机对话与用户自述想法配对：用户发送提示的原因以及对助手回复的反应。ThoughtTrace包含来自20个语言模型的1,058名用户、2,155次对话、17,058轮次和10,174条想法标注。我们的分析表明，ThoughtTrace捕捉了长期、主题多样的交互，且想法在语义上不同于消息，前沿LLM难以从上下文中推断，内容多样，并与对话阶段相关。我们进一步展示了想法在下游建模中的实用性。首先，想法作为推理时上下文改善了用户行为预测。其次，想法引导的重写为训练个性化助手提供了细粒度对齐信号。总之，ThoughtTrace将用户想法确立为研究人机交互背后认知动态的新数据模态，并为构建更好理解和适应用户潜在目标、偏好与需求的助手奠定了基础。
英文摘要：
Conversational AI has now reached billions of users, yet existing datasets capture only what people say, not what they think. We introduce ThoughtTrace, the first large-scale dataset that pairs real-world multi-turn human--AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses. ThoughtTrace comprises 1,058 users, 2,155 conversations, 17,058 turns, and 10,174 thought annotations collected across 20 language models. Our analysis shows that ThoughtTrace captures long-horizon, topically diverse interactions, and that thoughts are semantically distinct from messages, difficult for frontier LLMs to infer from context, diverse in content, and tied to conversation stages. We further demonstrate the utility of thoughts for downstream modeling. First, thoughts improve user-behavior prediction as inference-time context. Second, thought-guided rewrites provide fine-grained alignment signals for training personalized assistants. Together, ThoughtTrace establishes user thoughts as a new data modality for studying the cognitive dynamics behind human--AI interaction and provides a foundation for building assistants that better understand and adapt to users' latent goals, preferences, and needs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
