我们提出了Nautile-370M，一款3.7亿参数的小语言模型，旨在在严格的参数和推理预算下实现高效的推理。Nautile-370M采用混合主干结构，其中两个SeqCond Attention（SCA）层和一个线性时间频谱序列操作符（受SeqCondenser启发）交替使用一个Transformer层。这种设计旨在保留结构化序列模型的长上下文效率和状态跟踪优势，同时保持注意力机制的token到token路由能力。该模型在Google TPU Research Cloud（TRC）计划提供的单个Cloud TPU v4-64 pod slice上进行训练；随后的强化学习阶段在单个NVIDIA DGX Spark上进行。我们证明SCA读取机制可以精确检索前缀摘要中的任何单个token，并且可以将softmax注意力的任何输出作为特殊情况重现，从而证明SCA在连续极限下至少与完全自注意力一样具有表现力。我们还描述了训练数据管道，并概述了一个专门用于推理、验证和响应质量的强化学习阶段。
英文摘要：
We present Nautile-370M, a 371-million-parameter small language model designed for efficient reasoning under strict parameter and inference budgets. Nautile-370M uses a hybrid backbone in which two SeqCond Attention (SCA) layers, a linear-time spectral sequence operator inspired by SeqCondenser, alternate with one transformer layer. This design aims to retain the long-context efficiency and state-tracking benefits of structured sequential models while preserving the expressive token-to-token routing of attention. The model was trained on a single Cloud TPU v4-64 pod slice provided through the Google TPU Research Cloud (TRC) program; the subsequent reinforcement learning stage was carried out on a single NVIDIA DGX Spark. We prove that the SCA readout mechanism can exactly retrieve any individual token from the prefix summary and can reproduce any output of softmax attention as a special case, establishing that SCA is at least as expressive as full self-attention in the continuous limit. We also describe the training data pipeline and outline a reinforcement learning stage specialized for reasoning, verification, and response quality.
Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective
School of EEE, Nanyang Technological University（南洋理工大学电子工程系）；Department of Civil Engineering, Tsinghua University（清华大学土木工程系）；Dr. B. R. Ambedkar National Institute of Technology, Jalandhar（B.R.阿姆贝卡尔国家理工学院，贾兰德赫）；KTH Royal Institute of Technology（皇家理工学院）；College of AI, Tsinghua University（清华大学人工智能学院）；CCDS, Nanyang Technological University（南洋理工大学CCDS）
ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy
Tsinghua University（清华大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Institute of Intelligent Computing, Alibaba Group（阿里巴巴集团智能计算研究所）
In hierarchical planning for Markov decision processes (MDPs), temporal abstraction allows planning with macro-actions that take place at different time scale in form of sequential composition. In this paper, we propose a novel approach to compositional reasoning and hierarchical planning for MDPs under temporal logic constraints. In addition to sequential composition, we introduce a composition of policies based on generalized logic composition: Given sub-policies for sub-tasks and a new task expressed as logic compositions of subtasks, a semi-optimal policy, which is optimal in planning with…
The convergent laboratory: when AI reasoning, autonomous experiments, high performance and quantum computing reshape chemistry
Argonne National Laboratory（阿贡国家实验室）；The University of Chicago（芝加哥大学）；NVIDIA Corporation（英伟达公司）；Northwestern University（西北大学）；Lila Sciences, Inc.（Lila Sciences公司）；PsiQuantum（PsiQuantum公司）；University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（向量人工智能研究所）
At the age of big data, recommender systems have shown remarkable success as a key means of information filtering in our daily life. Recent years have witnessed the technical development of recommender systems, from perception learning to cognition reasoning which intuitively build the task of recommendation as the procedure of logical reasoning and have achieve significant improvement. However, the logical statement in reasoning implicitly admits irrelevance of ordering, even does not consider time information which plays an important role in many recommendation tasks. Furthermore, recommenda…
Retrospex: Language Agent Meets Offline Reinforcement Learning Critic
State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；Nanjing University School of Artificial Intelligence（南京大学人工智能学院）
Quantization Meets Reasoning: Exploring and Mitigating Degradation of Low-Bit LLMs in Mathematical Reasoning
The Hong Kong Polytechnic University（香港理工大学）；University of California, Santa Barbara（加州大学圣芭芭拉分校）；Peking University（北京大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；The University of Hong Kong（香港大学）
Stanford University（斯坦福大学）；Intel AI Lab（英特尔人工智能实验室）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Université de Montréal（蒙特利尔大学）
Complex logical query answering (CLQA) is a recently emerged task of graph machine learning that goes beyond simple one-hop link prediction and solves a far more complex task of multi-hop logical reasoning over massive, potentially incomplete graphs in a latent space. The task received a significant traction in the community; numerous works expanded the field along theoretical and practical axes to tackle different types of complex queries and graph modalities with efficient systems. In this paper, we provide a holistic survey of CLQA with a detailed taxonomy studying the field from multiple a…
Visual representation learning is ubiquitous in various real-world applications, including visual comprehension, video understanding, multi-modal analysis, human-computer interaction, and urban computing. Due to the emergence of huge amounts of multi-modal heterogeneous spatial/temporal/spatial-temporal data in big data era, the lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. The majority of the existing methods tend to fit the original data/variable distributions and ignore the essential causal relations b…
When Vision Meets Graphs: A Survey on Graph Reasoning and Learning
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Waterloo（滑铁卢大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Oxford（牛津大学）
GraphWalker: Patient Analogy Meets Information Gain for Clinical Reasoning with Large Language Models
School of Computer Science, Peking University, Beijing, China（北京大学计算机学院，北京，中国）；National Engineering Research Center for Software Engineering, Peking University, Beijing, China（软件工程国家工程研究中心，北京大学，北京，中国）
Virginia Tech（弗吉尼亚理工大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；US Army Research Laboratory（美国陆军研究实验室）；University of the District of Columbia（哥伦比亚特区大学）；University of Oslo（奥斯陆大学）
An in-depth understanding of uncertainty is the first step to making effective decisions under uncertainty. Deep/machine learning (ML/DL) has been hugely leveraged to solve complex problems involved with processing high-dimensional data. However, reasoning and quantifying different types of uncertainties to achieve effective decision-making have been much less explored in ML/DL than in other Artificial Intelligence (AI) domains. In particular, belief/evidence theories have been studied in KRR since the 1960s to reason and measure uncertainties to enhance decision-making effectiveness. We found…
Georgia Institute of Technology（佐治亚理工学院）；IBM Research（IBM研究院）
Discovering and exploiting the causality in deep neural networks (DNNs) are crucial challenges for understanding and reasoning causal effects (CE) on an explainable visual model. "Intervention" has been widely used for recognizing a causal relation ontologically. In this paper, we propose a causal inference framework for visual reasoning via do-calculus. To study the intervention effects on pixel-level features for causal reasoning, we introduce pixel-wise masking and adversarial perturbation. In our framework, CE is calculated using features in a latent space and perturbed prediction from a D…
Large Language Models Meet Text-Attributed Graphs: A Survey of Integration Frameworks and Applications
The University of New South Wales（新南威尔士大学）；The University of Technology Sydney（技术大学悉尼分校）；University of Technology Sydney（技术大学悉尼分校）；Duke University（杜克大学）
Lean Meets Theoretical Computer Science: Scalable Synthesis of Theorem Proving Challenges in Formal-Informal Pairs
D-CHAB, ETH Zurich, Zurich, Switzerland.；D-INFK, ETH Zurich, Zurich, Switzerland.；ETH AI Center, Zurich, Switzerland.；University of Pennsylvania, PA, USA.；Independent Researcher.
Language Agents Meet Causality -- Bridging LLMs and Causal World Models
University of Amsterdam（阿姆斯特丹大学）；Institute for Logic, Language and Computation (ILLC)（逻辑、语言与计算研究所）；University of Edinburgh（爱丁堡大学）；Institute for Language, Cognition and Computation (ILCC)（语言、认知与计算研究所）
