Large language models (LLMs) demonstrate the capacity to reconstruct and trace learned content from their training data under specific elicitation conditions, yet this capability does not manifest in standard generation contexts. This empirical observational study examines the expression of non-causal, non-implementable solution types across 300 prompt-response generations spanning narrative and problem-solving task contexts. Drawing on recent findings regarding memorization contiguity and alignment-induced discourse priors, we document a systematic dissociation between learned capability and expressed output. Across three distinct LLMs, ten task scenarios, and both creative narrative and practical advisory contexts, we documented zero instances of non-causal solution frames in generated outputs (0%, 95% CI: [0%, 1.2%]), despite verified reconstruction capability under conditional extraction. These findings challenge the prevailing assumption that training data presence directly predicts output probability, demonstrating instead that task-conditioned generation policies can comprehensively suppress learned content across diverse contexts. The results offer implications for understanding generation dynamics, output distribution control, and the behavioral boundaries of contemporary LLMs.
Towards Generalizable and Faithful Logic Reasoning over Natural Language via Resolution Refutation
Research Center for Social Computing and Information Retrieval(社会计算与信息检索研究中心);Harbin Institute of Technology(哈尔滨工业大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
针对 LLM 一阶逻辑推理的不完备问题,提出 GFaiR 框架,利用归结反演和反证法增强完备性,在复杂场景达到 SOTA 且推理过程忠实。
EduArt: An educational-level benchmark for evaluating art history knowledge in large language models
University of Bologna(博洛尼亚大学);Villa i Tatti – The Harvard University Center for Italian Renaissance Studies(哈佛大学意大利文艺复兴研究中心(I Tatti));University of Copenhagen(哥本哈根大学)
EvalTree: Profiling Language Model Weaknesses via Hierarchical Capability Trees
Paul G. Allen School of Computer Science & Engineering, University of Washington(保罗·G·艾伦计算机科学与工程学院,华盛顿大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所)
Multilingual GSM-Symbolic: What determines capability transfer across languages?
Aarhus University(奥胡斯大学);Danish Foundation Models(丹麦基础模型公司);University of Alabama(阿拉巴马大学);Alexandra Institute(亚历山德拉研究所);Indian Institute of Technology Kharagpur(印度理工学院卡拉格普尔分校);The University of Tokyo(东京大学);IT University of Copenhagen(哥本哈根信息技术大学);Massachusetts General Hospital(马萨诸塞州总医院);Bocconi University(博科尼大学);University of Southern Denmark(南丹麦大学);University of Iceland(冰岛大学);University of the Faroe Islands(法罗群岛大学);Zendesk(Zendesk公司);University of Copenhagen(哥本哈根大学);Indian Institute of Technology Madras(印度理工学院马德拉斯分校);National Library of Sweden(瑞典国家图书馆)
Many Dialects, Many Languages, One Cultural Lens: Evaluating Multilingual VLMs for Bengali Culture Understanding Across Historically Linked Languages and Regional Dialects
United International University(国际联合大学);BRAC University(布拉塔克大学);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校)