$A^3$-Bench:通过锚点和吸引子激活进行记忆驱动科学推理的基准测试
$A^3$-Bench: Benchmarking Memory-Driven Scientific Reasoning via Anchor and Attractor Activation
AI总结:
$A^3$-Bench通过双尺度记忆驱动激活机制,评估科学推理中的记忆利用效果,揭示记忆激活对推理性能的影响。
摘要:
科学推理不仅依赖于逻辑推理,还依赖于激活先前知识和经验结构。记忆可以高效地重用知识,提高推理的一致性和稳定性。然而,现有基准主要评估最终答案或逐步连贯性,忽略了人类推理背后记忆驱动的机制,即激活锚点和吸引子,然后将它们整合到多步骤推理中。为了解决这一差距,我们提出了$A^3$-Bench~ https://a3-bench.github.io,一个旨在通过双尺度记忆驱动激活评估科学推理的基准,基于锚点和吸引子激活。首先,我们使用SAPM过程(主题、锚点与吸引子、问题和记忆发展)对跨领域的2198个科学推理问题进行注释。其次,我们引入一个双尺度记忆评估框架,利用锚点和吸引子,以及AAUI(锚点-吸引子利用指数)指标来衡量记忆激活率。最后,通过各种基础模型和范式进行实验,我们验证了$A^3$-Bench,并分析了记忆激活对推理性能的影响,为记忆驱动的科学推理提供了见解。
英文摘要:
Scientific reasoning relies not only on logical inference but also on activating prior knowledge and experiential structures. Memory can efficiently reuse knowledge and enhance reasoning consistency and stability. However, existing benchmarks mainly evaluate final answers or step-by-step coherence, overlooking the \textit{memory-driven} mechanisms that underlie human reasoning, which involves activating anchors and attractors, then integrating them into multi-step inference. To address this gap, we propose $A^3$-Bench~ https://a3-bench.github.io, a benchmark designed to evaluate scientific reasoning through dual-scale memory-driven activation, grounded in Anchor and Attractor Activation. First, we annotate 2,198 science reasoning problems across domains using the SAPM process(subject, anchor & attractor, problem, and memory developing). Second, we introduce a dual-scale memory evaluation framework utilizing anchors and attractors, along with the AAUI(Anchor--Attractor Utilization Index) metric to measure memory activation rates. Finally, through experiments with various base models and paradigms, we validate $A^3$-Bench and analyze how memory activation impacts reasoning performance, providing insights into memory-driven scientific reasoning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。