arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.02536cs.AIcs.NE
收藏

混沌边缘的智能

Intelligence at the Edge of Chaos

Yale University(耶鲁大学);Columbia University(哥伦比亚大学);Northwestern University(西北大学);Idaho State University(爱达荷州立大学)

Shiyang Zhang, Aakash Patel, Syed A Rizvi, Nianchen Liu, Sizhuang He, Amin Karbasi, Emanuele Zappala, David van Dijk

AI总结:

本文通过在基本元胞自动机上训练LLM,发现规则复杂性与模型智能呈正相关,且存在最有利于智能涌现的复杂性最佳点,推测智能源于预测复杂性的能力。

摘要:

我们通过研究基于规则的系统的复杂性如何影响被训练来预测这些规则的模型的能力,探索了人工系统中智能行为的涌现。我们的研究聚焦于基本元胞自动机(ECA),这是一种简单却强大的一维系统,能够产生从平凡到高度复杂的行为。通过在不同的ECA上训练不同的大型语言模型(LLM),我们评估了规则行为的复杂性与LLM所展现的智能之间的关系,这种关系反映在它们在下游任务上的表现中。我们的发现表明,具有更高复杂性的规则会导致模型展现出更高的智能,正如它们在推理和国际象棋走子预测任务上的表现所证明的那样。均匀和周期性系统,以及通常的高度混沌系统,都导致了较差的下游性能,突出了一个有利于智能涌现的复杂性最佳点。我们推测,智能源于预测复杂性的能力,并且创造智能可能只需要暴露于复杂性之中。

英文摘要:

We explore the emergence of intelligent behavior in artificial systems by investigating how the complexity of rule-based systems influences the capabilities of models trained to predict these rules. Our study focuses on elementary cellular automata (ECA), simple yet powerful one-dimensional systems that generate behaviors ranging from trivial to highly complex. By training distinct Large Language Models (LLMs) on different ECAs, we evaluated the relationship between the complexity of the rules' behavior and the intelligence exhibited by the LLMs, as reflected in their performance on downstream tasks. Our findings reveal that rules with higher complexity lead to models exhibiting greater intelligence, as demonstrated by their performance on reasoning and chess move prediction tasks. Both uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence. We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑