无需持续监督的代理：挑战与机遇
Overseeing Agents Without Constant Oversight: Challenges and Opportunities
AI总结：
本研究探讨了无需持续监督的代理系统在验证中的挑战与机遇，通过实验发现简化轨迹设计可提高错误发现效率，但未显著提升准确性。
摘要：
为实现人类监督，代理AI系统通常会提供推理和行动步骤的轨迹。设计轨迹以具有信息性但不令人压倒性的细节水平仍然是一个关键挑战。在一项针对计算机用户代理的三项用户研究中，我们调查了基本行动轨迹在验证中的效用，通过设计探测探索了三种替代方案，并测试了新界面在问答任务中发现错误的影响。正如预期的那样，我们发现当前的做法繁琐，限制了其有效性。相反，我们提出的设计减少了参与者发现错误所花的时间。然而，尽管参与者报告了更高的决策信心水平，但他们的最终准确性并未显著提高。为此，我们的研究揭示了人类验证代理系统面临的挑战，包括管理内置假设、用户主观和变化的正确性标准，以及传达代理过程的重要性与不足。
英文摘要：
To enable human oversight, agentic AI systems often provide a trace of reasoning and action steps. Designing traces to have an informative, but not overwhelming, level of detail remains a critical challenge. In three user studies on a Computer User Agent, we investigate the utility of basic action traces for verification, explore three alternatives via design probes, and test a novel interface's impact on error finding in question-answering tasks. As expected, we find that current practices are cumbersome, limiting their efficacy. Conversely, our proposed design reduced the time participants spent finding errors. However, although participants reported higher levels of confidence in their decisions, their final accuracy was not meaningfully improved. To this end, our study surfaces challenges for human verification of agentic systems, including managing built-in assumptions, users' subjective and changing correctness criteria, and the shortcomings, yet importance, of communicating the agent's process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
