评估人工智能欺骗检测器的困难
Difficulties with Evaluating a Deception Detector for AIs
AI总结：
本文探讨了评估人工智能欺骗检测器的困难，指出缺乏足够标记数据和收集障碍，并分析了现有研究和潜在解决方案的局限性。
摘要：
构建可靠的欺骗检测器对于AI系统至关重要——这些方法能够预测AI系统在策略上欺骗而无需行为证据的情况——在缓解高级AI系统风险方面具有价值。但评估所提出欺骗检测器的可靠性和有效性需要我们能够自信地标记为欺骗或诚实的例子。我们主张目前缺乏必要的例子，并进一步识别了收集它们的几个具体障碍。我们提供了概念论证、现有实证研究的分析以及新示例案例研究的分析。我们还讨论了几种提出的实证解决方案的潜力，并认为尽管它们似乎有价值，但单独使用似乎不足。对欺骗检测的进展可能需要进一步考虑这些问题。
英文摘要：
Building reliable deception detectors for AI systems -- methods that could predict when an AI system is being strategically deceptive without necessarily requiring behavioural evidence -- would be valuable in mitigating risks from advanced AI systems. But evaluating the reliability and efficacy of a proposed deception detector requires examples that we can confidently label as either deceptive or honest. We argue that we currently lack the necessary examples and further identify several concrete obstacles in collecting them. We provide evidence from conceptual arguments, analysis of existing empirical works, and analysis of novel illustrative case studies. We also discuss the potential of several proposed empirical workarounds to these problems and argue that while they seem valuable, they also seem insufficient alone. Progress on deception detection likely requires further consideration of these problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
