DRIP-R：零售领域决策与推理在现实政策模糊性下的基准测试
DRIP-R: A Benchmark for Decision-Making and Reasoning Under Real-World Policy Ambiguity in the Retail Domain
AI总结：
DRIP-R基准测试通过现实零售政策模糊性构建无唯一正确解决方案的场景，评估LLM在模糊政策下的决策与推理能力，揭示其固有的系统性挑战。
摘要：
基于LLM的智能体正被用于现实领域中的常规但重要的任务，其行为由本质上模糊的领域政策所支配，这些政策允许多种合理的解释。尽管现实中存在此类模糊性，现有智能体基准测试大多假设政策明确且规范，留下关键评估空白。我们引入DRIP-R基准测试，系统利用现实零售政策模糊性构建场景，其中不存在单一正确解决方案。DRIP-R包含一个精心编纂的政策模糊退货场景集，配以现实客户画像，一个具有工具调用能力的全双工对话模拟，以及涵盖政策合规性、对话质量、行为一致性及解决方案质量的多评委评估框架。我们的实验表明，前沿模型在相同政策模糊场景下根本性地存在分歧，证实模糊性对LLM决策构成了真实且系统性的挑战。
英文摘要：
LLM-based agents are increasingly deployed for routine but consequential tasks in real-world domains, where their behavior is governed by inherently ambiguous domain policies that admit multiple valid interpretations. Despite the prevalence of such ambiguities in practice, existing agent benchmarks largely assume unambiguous, well-specified policies, leaving a critical evaluation gap. We introduce DRIP-R, a benchmark that systematically exploits real-world retail policy ambiguities to construct scenarios in which no single correct resolution exists. DRIP-R comprises a curated set of policy-ambiguous return scenarios paired with a realistic customer personas, a full-duplex conversational simulation with tool-calling capabilities and a multi-judge evaluation framework covering policy adherence, dialogue quality, behavioral alignment, and resolution quality. Our experiments show that frontier models fundamentally disagree on identical policy-ambiguous scenarios, confirming that ambiguity poses a genuine and systematic challenge to LLM decision-making.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
