Humans can apply ideas learned in one context to substantially different situations. We call this process of searching for and constructing novel recombinations of learned relationships to solve new problems \textit{out-of-distribution (OOD) reasoning}. The capacity for large language models (LLMs) to support OOD reasoning underpins proposals for generally intelligent systems. However, this property is challenging to measure because most problems admit many decompositions, some involving shallow subproblems and others involving subproblems that may have been memorized from the training data. This makes it difficult to determine how much compositional reasoning a model must actually perform. Uncertainty about training distributions, how to measure a datapoint's distance from a training distribution, and the exponential number of ways to decompose most tasks make it intractable to robustly measure any modern LLM's OOD-reasoning capacity on standard natural-language tasks. In this work, we introduce a benchmark in which a model progressively solves a Boolean circuit over $GF(2)$ from data. This benchmark is both minimal, isolating OOD reasoning from these confounding factors, and general, as any computable function can be represented as a sufficiently large $GF(2)$ polynomial. We find that standalone models' next-step accuracy collapses as depth grows. In contrast, tool use through the synthesis and execution of code prevents this collapse in both small and frontier LLMs. These results indicate that synthesizing tools play a crucial role in supporting OOD reasoning.
Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection
University of Surrey(萨里大学);University of Edinburgh(爱丁堡大学)
由于在实际应用中部署机器学习模型的重要性,分发外(OOD)检测最近受到了机器学习社区的广泛关注。在本文中,我们提出了一个不确定性量化的方法,通过建模的分布特征。我们进一步结合了一个有效的集成机制,即批量集成,构建批量集成的随机神经网络(BE-SNNs),并克服了功能崩溃的问题。我们将所提出的BE-SNN的性能与其他最先进的方法进行了比较,并表明BE-SNN在几个OOD基准上具有优异的性能,例如Two-Moons数据集,FashionMNIST vs MNIST数据集,FashionMNIST vs NotMNIST数据集以及CIFAR 10 vs SVHN数据集。
Composition Collapse: Stable Factual Knowledge Does Not Imply Compositional Reasoning
Zhejiang University(浙江大学);Binjiang Institute of Zhejiang University(浙江大学滨江研究院);Hong Kong Baptist University(香港 Baptist大学);Harbin Institute of Technology(哈尔滨工业大学);Hangzhou Dianzi University(杭州电子科技大学)
On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective
Microsoft Research(微软研究院);City University of Hong Kong(香港城市大学);Microsoft STCA(微软中国区软件工程技术中心);Carnegie Mellon University(卡内基梅隆大学);Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳));Peking University(北京大学);Westlake University(西湖大学)