MCTSr-Zero：基于原则与自适应探索的自反思心理咨询对话生成
MCTSr-Zero: Self-Reflective Psychological Counseling Dialogues Generation via Principles and Adaptive Exploration
AI总结：
针对现有MCTS方法在心理咨询等开放式对话中易生成错位响应的问题，提出MCTSr-Zero框架，通过领域对齐、再生和元提示自适应机制优化探索，生成的对话数据微调的PsyLLM在PsyEval等基准上达最优性能。
摘要：
蒙特卡洛树搜索（MCTS）与大型语言模型（LLMs）的集成在结构化、面向问题的任务中已展现显著成功。然而，将这些方法应用于开放式对话（如心理咨询对话）面临独特挑战。与具有客观正确性的任务不同，治疗对话的成功取决于共情参与、伦理遵守和与人类偏好的一致性等主观因素，这些因素缺乏明确的“正确性”标准。因此，现有结果导向的MCTS方法可能生成错位的响应。为解决此问题，我们提出MCTSr-Zero，一种为开放式、以人为中心的对话设计的MCTS框架。其核心创新是“领域对齐”，将MCTS的搜索目标从预定义的终态转向符合目标领域原则（如咨询中的共情）的对话轨迹。此外，MCTSr-Zero整合“再生”和“元提示自适应”机制，通过允许MCTS考虑根本不同的初始对话策略来大幅拓宽探索。我们在心理咨询中评估MCTSr-Zero，生成多轮对话数据用于微调LLM（PsyLLM），还引入PsyEval基准评估多轮心理咨询对话。实验表明，PsyLLM在PsyEval和其他相关指标上达到最先进性能，验证了MCTSr-Zero在生成高质量、原则对齐的以人为中心领域对话数据及解决LLM持续遵守复杂心理标准挑战方面的有效性。
英文摘要：
The integration of Monte Carlo Tree Search (MCTS) with Large Language Models (LLMs) has demonstrated significant success in structured, problem-oriented tasks. However, applying these methods to open-ended dialogues, such as those in psychological counseling, presents unique challenges. Unlike tasks with objective correctness, success in therapeutic conversations depends on subjective factors like empathetic engagement, ethical adherence, and alignment with human preferences, for which strict "correctness" criteria are ill-defined. Existing result-oriented MCTS approaches can therefore produce misaligned responses. To address this, we introduce MCTSr-Zero, an MCTS framework designed for open-ended, human-centric dialogues. Its core innovation is "domain alignment", which shifts the MCTS search objective from predefined end-states towards conversational trajectories that conform to target domain principles (e.g., empathy in counseling). Furthermore, MCTSr-Zero incorporates "Regeneration" and "Meta-Prompt Adaptation" mechanisms to substantially broaden exploration by allowing the MCTS to consider fundamentally different initial dialogue strategies. We evaluate MCTSr-Zero in psychological counseling by generating multi-turn dialogue data, which is used to fine-tune an LLM, PsyLLM. We also introduce PsyEval, a benchmark for assessing multi-turn psychological counseling dialogues. Experiments demonstrate that PsyLLM achieves state-of-the-art performance on PsyEval and other relevant metrics, validating MCTSr-Zero's effectiveness in generating high-quality, principle-aligned conversational data for human-centric domains and addressing the LLM challenge of consistently adhering to complex psychological standards.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
