Agentic AI与人在回路干预：来自阿里巴巴客服运营的现场实验证据
Agentic AI and Human-in-the-Loop Interventions: Field Experimental Evidence from Alibaba's Customer Service Operations
AI总结：
通过在阿里巴巴淘宝平台上的随机现场实验，研究Agentic AI在客服运营中失败时，人类干预如何影响服务结果，发现干预效果取决于AI失败类型、干预时机和努力程度。
摘要：
自主执行服务任务的Agentic AI系统正进入客服运营。然而，当Agentic AI失败产生认知和情感后果时，人类干预如何塑造服务结果，现有证据有限。我们通过在阿里巴巴淘宝平台上的随机现场实验研究这一问题。处理组工人监督一个Agentic AI系统，该系统解决AI可处理的聊天，同时继续处理AI不可处理的聊天，而对照组工人处理所有聊天，没有Agentic AI。结果显示，AI部署减少了平均聊天时长，对重试率影响有限，但显著降低了AI可处理聊天的评分。此外，在AI可处理聊天中，人类干预的有效性取决于AI失败的性质、升级后干预努力和干预时机。人类干预在算法触发的技术升级（即超出AI能力的未解决客户问题）中保持了服务质量，但在算法触发的情感升级（即客户表达沮丧或不满）中效果较差。这些差异部分由工人在不同升级类型中的升级后干预努力变化解释。在算法触发的情感升级中，工人表现出较低的参与度：他们发送更少消息，贡献更小比例的聊天轮次，并且在信息寻求和解决方案提供方面表现出较少的主动性。我们进一步发现，早期干预对于维持高水平的升级后干预努力至关重要。最后，我们记录了对AI不可处理聊天的正向溢出效应，因为处理组工人调整了多任务处理流程，将更多注意力投入到这些聊天中。这些发现为人类-AI协作系统中的人在回路流程设计提供了启示。
英文摘要：
Agentic AI systems that autonomously perform service tasks are entering customer service operations. However, limited evidence exists on how human interventions shape service outcomes when agentic AI failures create both cognitive and emotional consequences. We study this issue through a randomized field experiment on Alibaba's Taobao platform. Workers in the treatment condition supervised an agentic AI system that resolved AI-eligible chats while continuing to handle AI-ineligible chats, whereas control workers resolved all chats without agentic AI. The findings show that AI deployment reduces average chat duration and has limited effects on retrial rates, but substantially lowers ratings for AI-eligible chats. Moreover, human intervention effectiveness in AI-eligible chats depends on the nature of AI failure, post-escalation intervention effort, and intervention timing. Human intervention preserves service quality in algorithm-triggered technical escalations, i.e., unresolved customer issues beyond the AI's capability, but is less effective in algorithm-triggered emotional escalations, i.e., where customers express frustration or dissatisfaction. These differences are partly explained by variation in workers' post-escalation intervention effort across escalation types. In algorithm-triggered emotional escalations, workers showed lower engagement: they sent fewer messages, contributed a smaller share of total chat rounds, and showed less proactivity in information seeking and solution provision. We further find that early intervention is essential for sustaining high post-escalation intervention effort. Finally, we document a positive spillover effect on AI-ineligible chats, as treated workers adapted their multitasking workflow to devote greater attention to these chats. These findings offer implications for human-in-the-loop process design in human-AI collaboration systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
