TripScore：通过专家校准奖励使大语言模型与真实旅行规划对齐
TripScore: Aligning LLMs for Real-World Travel Planning via Expert-Calibrated Reward
AI总结：
TripScore是一个基于真实用户日志和专家校准的旅行规划评估框架，通过分层可行性门控与统一奖励，发现强化学习微调（如GRPO）在相同基础模型下优于其他方法。
摘要：
在我们已部署的旅行规划服务中，大多数用户提供的是极简输入或自由形式的请求，而非现有基准所假设的结构化约束清单。为此，我们提出TripScore，一个基于真实用户日志构建、并以203位旅行专家的1,468对成对判断进行校准的行为导向基准与评估框架。TripScore将分层可行性门控（格式与常识）与统一的逐点奖励相结合，该奖励聚合了软质量与偏好满足度。我们将TripScore同时用作评估器和奖励信号，对直接提示、测试时计算、神经符号求解器、代码智能体以及微调进行了基准测试。我们发现，在相同基础模型和实际延迟条件下，强化学习微调（如GRPO）相较于其他方法提供了持续的性能提升。
英文摘要：
In our deployed travel-planning service, most users give minimal inputs or free-form requests rather than the structured constraint checklists assumed by existing benchmarks. We therefore present TripScore, a behavior-grounded benchmark and evaluation framework built from real user logs and calibrated against 1,468 pairwise judgments by 203 travel experts. TripScore couples a hierarchical feasibility gate (format and commonsense) with a unified, point-wise reward that aggregates soft quality and preference fulfillment. Using TripScore as both evaluator and reward signal, we benchmark direct prompting, test-time compute, neuro-symbolic solvers, code agents, and fine-tuning. We find that reinforcement learning fine-tuning (e.g., GRPO) provides consistent gains over other approaches under the same base model and practical latency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
