arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.10327cs.AIcs.CLcs.HChep-phphysics.hist-ph
收藏

迈向人类与机器科学理解的基准

Towards a Benchmark for Scientific Understanding in Humans and Machines

Institute for Science in Society, Faculty of Science, Radboud University(拉德堡德大学理学院社会中的科学研究所);High Energy Physics, Faculty of Science, Radboud University(拉德堡德大学理学院高能物理系);Nikhef(荷兰国家亚原子物理研究所);Institute for Computing and Information Sciences, Faculty of Science, Radboud University(拉德堡德大学理学院计算与信息科学研究所)

Kristian Gonzalez Barman, Sascha Caron, Tom Claassen, Henk de Regt

AI总结:

本文提出科学理解基准(SUB)框架,利用科学哲学工具设计覆盖信息检索、解释组织和反事实推断的测试,以评估和比较人类与机器的科学理解能力,从而推动科学理解发展。

摘要:

科学理解是科学的一个基本目标,使我们能够解释世界。目前没有好的方法来衡量智能体的科学理解,无论这些智能体是人类还是人工智能系统。没有明确的基准,评估和比较不同水平的科学理解以及不同的科学理解方法都具有挑战性。在本路线图中,我们提出了一个框架,利用科学哲学的工具来创建科学理解基准。我们采用一种行为主义的概念,根据这一概念,真正的理解应被认定为执行某些任务的能力。我们通过考虑一组能够衡量不同水平科学理解的问题来扩展这一概念,这些问题涵盖信息检索、组织信息以产生解释的能力,以及推断在不同情况下事物会如何不同的能力。科学理解基准(SUB)由一组这样的测试构成,允许对不同方法进行评估和比较。基准测试在建立信任、确保质量控制和为性能评估提供基础方面发挥着关键作用。通过将机器和人类的科学理解对齐,我们可以提高它们的效用,最终推进科学理解,并帮助在机器中发现新的见解。

英文摘要:

Scientific understanding is a fundamental goal of science, allowing us to explain the world. There is currently no good way to measure the scientific understanding of agents, whether these be humans or Artificial Intelligence systems. Without a clear benchmark, it is challenging to evaluate and compare different levels of and approaches to scientific understanding. In this Roadmap, we propose a framework to create a benchmark for scientific understanding, utilizing tools from philosophy of science. We adopt a behavioral notion according to which genuine understanding should be recognized as an ability to perform certain tasks. We extend this notion by considering a set of questions that can gauge different levels of scientific understanding, covering information retrieval, the capability to arrange information to produce an explanation, and the ability to infer how things would be different under different circumstances. The Scientific Understanding Benchmark (SUB), which is formed by a set of these tests, allows for the evaluation and comparison of different approaches. Benchmarking plays a crucial role in establishing trust, ensuring quality control, and providing a basis for performance evaluation. By aligning machine and human scientific understanding we can improve their utility, ultimately advancing scientific understanding and helping to discover new insights within machines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑