评估：停止用人类测试评估AI，改而开发基于AI的原理性测试
Position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead
AI总结：
本文指出，将AI性能评估等同于人类心理特质是理论和实证上的错误，呼吁开发专门针对AI的评估框架。
摘要：
大型语言模型（LLMs）在一系列原本用于评估人类认知和心理特质的标准测试上取得了显著成果，如智力和性格。尽管这些结果常被解释为LLMs具有类人特征的证据，本文认为这种解释是一种本体论错误。人类心理和教育测试是理论驱动的测量工具，校准于特定的人群。将这些测试应用于非人类主体而无实证验证，可能歪曲所测量的内容。此外，一种趋势将AI在基准测试上的表现视为智力等特质的测量，尽管已知存在有效性、数据污染、文化偏见和对表面提示变化的敏感性问题。本文认为，将基准性能解释为类人特质的测量缺乏充分的理论和实证依据。因此，我们的立场是：停止用人类测试评估AI，改而开发基于AI的原理性测试。我们呼吁开发专门针对AI的评估框架，这些框架可能基于现有心理测量测试的构建和验证框架，或完全从头开始设计，以适应AI的独特背景。
英文摘要：
Large Language Models (LLMs) have achieved remarkable results on a range of standardized tests originally designed to assess human cognitive and psychological traits, such as intelligence and personality. While these results are often interpreted as strong evidence of human-like characteristics in LLMs, this paper argues that such interpretations constitute an ontological error. Human psychological and educational tests are theory-driven measurement instruments, calibrated to a specific human population. Applying these tests to non-human subjects without empirical validation, risks mischaracterizing what is being measured. Furthermore, a growing trend frames AI performance on benchmarks as measurements of traits such as ``intelligence'', despite known issues with validity, data contamination, cultural bias and sensitivity to superficial prompt changes. We argue that interpreting benchmark performance as measurements of human-like traits, lacks sufficient theoretical and empirical justification. This leads to our position: Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead. We call for the development of principled, AI-specific evaluation frameworks tailored to AI systems. Such frameworks might build on existing frameworks for constructing and validating psychometrics tests, or could be created entirely from scratch to fit the unique context of AI.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
