噪声对LLM模型在抽象与推理语料库（ARC）任务中性能的影响及模型温度考量
Impact of Noise on LLM-Models Performance in Abstraction and Reasoning Corpus (ARC) Tasks with Model Temperature Considerations
AI总结：
本研究系统评估了噪声和温度设置对GPT-4o、DeepSeek R1和LLaMA 3.2在ARC任务上性能的影响，发现噪声持续损害所有模型性能，揭示了当前LLM在抽象推理中对输入扰动的高度脆弱性，强调了开发更鲁棒AI系统的必要性。
摘要：
大型语言模型（LLM）的最新进展引发了人们对其结构化推理能力的日益关注，特别是在涉及抽象和模式识别的任务中。抽象与推理语料库（ARC）基准通过测试AI模型对新颖问题的泛化能力，在评估这些能力方面发挥着关键作用。虽然GPT-4o在零噪声条件下通过解决所有ARC任务展现出强劲性能，但DeepSeek R1和LLaMA 3.2等其他模型未能解决任何任务，表明它们在超越简单模式匹配的推理能力上存在局限。为探究这一差距，我们系统性地评估了这些模型在不同噪声水平和温度设置下的表现。我们的结果显示，无论架构如何，引入噪声都会持续损害模型性能。这种下降凸显了一个共同的脆弱性：当前的LLM尽管显示出抽象推理的迹象，但对输入扰动仍然高度敏感。这种脆弱性引发了对它们在现实世界适用性的担忧，因为噪声和不确定性在现实世界中很常见。通过比较不同模型架构对这些挑战的响应，我们提供了对现代LLM在推理任务中结构性弱点的见解。这项工作强调了开发更鲁棒、更自适应的AI系统的必要性，这些系统能够处理现实场景中固有的模糊性和变异性。我们的发现旨在引导未来研究，以增强模型泛化能力、鲁棒性以及与人类认知灵活性的一致性。
英文摘要：
Recent advancements in Large Language Models (LLMs) have generated growing interest in their structured reasoning capabilities, particularly in tasks involving abstraction and pattern recognition. The Abstraction and Reasoning Corpus (ARC) benchmark plays a crucial role in evaluating these capabilities by testing how well AI models generalize to novel problems. While GPT-4o demonstrates strong performance by solving all ARC tasks under zero-noise conditions, other models like DeepSeek R1 and LLaMA 3.2 fail to solve any, suggesting limitations in their ability to reason beyond simple pattern matching. To explore this gap, we systematically evaluate these models across different noise levels and temperature settings. Our results reveal that the introduction of noise consistently impairs model performance, regardless of architecture. This decline highlights a shared vulnerability: current LLMs, despite showing signs of abstract reasoning, remain highly sensitive to input perturbations. Such fragility raises concerns about their real-world applicability, where noise and uncertainty are common. By comparing how different model architectures respond to these challenges, we offer insights into the structural weaknesses of modern LLMs in reasoning tasks. This work underscores the need for developing more robust and adaptable AI systems capable of handling the ambiguity and variability inherent in real-world scenarios. Our findings aim to guide future research toward enhancing model generalization, robustness, and alignment with human-like cognitive flexibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
