Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models
Comments v3.0. Control experiments, further AIW problem versions, testing recent reasoning models. Short version appeared at NeurIPS Scientific Methods for Understanding Deep Learning Workshop (SciDL) 2024, https://openreview.net/forum?id=Mkl7dzjYiW