关于注意力反向传递中细粒度I/O复杂度

量化预训练的混合精度缩放定律

Mixed Precision Scaling Laws for Quantized Pretraining

本文针对量化预训练提出混合精度缩放定律，分离量化对模型容量与优化效率的影响，经Qwen3、Llama实验验证，其预测损失的MSE比现有定律低达11倍，还确定了硬件感知下的最优量化预训练方案。

无需反向传播的节能深度学习：对仅前向算法的严格评估

Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Hardware-Validated Benchmarking Study of Forward-Only Algorithms

本研究挑战了反向传播不可或缺的假设，证明无反向传播的Mono-Forward算法在MLP架构上分类准确率超越BP基线，且能耗降低41%、训练加速34%，是替代BP的高性能可持续方案。

关注生成过程：LLM生成过程中的细粒度置信度估计

Mind the Generation Process: Fine-Grained Confidence Estimation During LLM Generation

本文提出FineCE方法，通过构建捕捉LLM响应概率分布的训练流程和监督学习，结合后向置信度集成策略及最佳估计位置选择，在生成过程中实现细粒度、连续的置信度估计，并在多个基准上优于现有方法。

基于结果锚定的优势重塑用于数学推理中的细粒度信用分配

Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning

提出结果锚定优势重塑（OAR），通过两种策略（OAR-P和OAR-G）实现细粒度信用分配，显著提升GRPO在数学推理中的性能。

BAS-OPD：面向细粒度多模态感知的预算感知选择性在线策略自蒸馏

BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception

针对多模态大模型细粒度感知中教师查询成本高的问题，提出预算感知的选择性在线策略自蒸馏框架BAS-OPD，通过智能选择信息样本分配监督，在保持性能的同时大幅降低训练成本。

FZOO：面向Adam级速度的大语言模型微调快速零阶优化器

FZOO: Fast Zeroth-Order Optimizer for Fine-Tuning Large Language Models towards Adam-Scale Speed

针对大语言模型微调中的内存瓶颈，提出快速零阶优化器FZOO，通过批量单侧估计和Rademacher扰动减少前向传播次数并加速计算，在准确率和收敛速度上显著优于MeZO，实现单GPU全参数微调。

面向设备上大语言模型微调的内存高效结构化反向传播

Memory-Efficient Structured Backpropagation for On-Device LLM Fine-Tuning

提出MeSP方法，通过手动推导利用LoRA低秩结构的反向传播，在计算数学等价梯度的同时平均减少49%内存，使内存受限设备上的微调成为可能。

差异中的和谐：迈向快速、准确且内存高效的零阶LLM微调

Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuning

为解决零阶优化在LLM微调中收敛慢、精度低的问题，提出差异驱动的DiZO方法，通过逐层差异分析引入投影生成多样化幅度更新，显著提升收敛速度与精度，并大幅降低GPU训练时长。

Abnormal activity capture from passenger flow of elevator based on unsupervised learning and fine-grained multi-label recognition

We present a work-flow which aims at capturing residents' abnormal activities through the passenger flow of elevator in multi-storey residence buildings. Camera and sensors (hall sensor, photoelectric sensor, gyro, accelerometer, barometer, and thermometer) with internet connection are mounted in elevator to collect image and data. Computer vision algorithms such as instance segmentation, multi-label recognition, embedding and clustering are applied to generalize passenger flow of elevator, i.e. how many people and what kinds of people get in and out of the elevator on each floor. More specifi…

One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning

Deep learning models in large-scale machine learning systems are often continuously trained with enormous data from production environments. The sheer volume of streaming training data poses a significant challenge to real-time training subsystems and ad-hoc sampling is the standard practice. Our key insight is that these deployed ML systems continuously perform forward passes on data instances during inference, but ad-hoc sampling does not take advantage of this substantial computational effort. Therefore, we propose to record a constant amount of information per instance from these forward p…

StreamKL: 快速且内存高效的KL散度用于提升注意力蒸馏

StreamKL: Fast and Memory-Efficient KL Divergence for Boosting Attention Distillation

提出StreamKL，首个融合GPU原语，通过在线公式和逐块重计算将注意力蒸馏的内存和IO成本从O(N_QN_K)降至O(1)，实现高达43倍前向和14倍反向加速。

FoodSky：一个通过厨师与营养师考试的面向食品的大语言模型

FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examination

本文提出面向食品的大语言模型 FoodSky，结合中文食品语料库 FoodEarth、TS3M 与 HTRAG，提升食品语义理解和上下文生成能力，并在国家厨师及营养师考试中显著超越通用 LLM。

通过近似反向传播与内存共享反向传播降低微调内存开销

Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation

针对大模型微调内存开销大的问题，提出近似反向传播理论与内存共享反向传播策略，在不增加计算、不降低效率的前提下，最多可降低约30%的峰值内存使用量。

反向透镜：将语言模型梯度投影到词汇空间

Backward Lens: Projecting Language Model Gradients into the Vocabulary Space

本文将语言模型可解释性从前向传播扩展到反向传播，证明梯度可表示为前后向输入的秩受限线性组合，并提出将梯度投影到词汇空间的方法，以揭示新信息在神经元中的存储机制。

RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching

Activation patching is a standard method in mechanistic interpretability for localizing the components of a model responsible for specific behaviors, but it is computationally expensive to apply at scale. Attribution patching offers a faster, gradient-based approximation, yet suffers from noise and reduced reliability in deep, highly non-linear networks. In this work, we introduce Relevance Patching (RelP), which replaces the local gradients in attribution patching with propagation coefficients derived from Layer-wise Relevance Propagation (LRP). LRP propagates the network's output backward th…

构建逆向思维：发展大语言模型的逆向思考能力

Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Ability

针对大模型前向推理的局限，提出逆向推理模式构建方法，通过两阶段数据集与微调、细粒度奖励机制及平衡采样，显著提升数学证明等任务的推理效率与准确性。

LMC：通过具有可证明收敛性的子图采样实现 GNN 快速训练

LMC: Fast Training of GNNs via Subgraph Sampling with Provable Convergence

针对大规模 GNN 训练中的邻居爆炸与子图采样梯度偏差问题，本文提出可证明收敛的 LMC，通过前后向消息补偿获得准确小批量梯度并提升效率。

Spike-based causal inference for weight alignment

In artificial neural networks trained with gradient descent, the weights used for processing stimuli are also used during backward passes to calculate gradients. For the real brain to approximate gradients, gradient information would have to be propagated separately, such that one set of synaptic weights is used for processing and another set is used for backward passes. This produces the so-called "weight transport problem" for biological models of learning, where the backward weights used to calculate gradients need to mirror the forward weights used to process stimuli. This weight transport…

The Forward-Forward Algorithm: Some Preliminary Investigations

The aim of this paper is to introduce a new learning procedure for neural networks and to demonstrate that it works well enough on a few small problems to be worth further investigation. The Forward-Forward algorithm replaces the forward and backward passes of backpropagation by two forward passes, one with positive (i.e. real) data and the other with negative data which could be generated by the network itself. Each layer has its own objective function which is simply to have high goodness for positive data and low goodness for negative data. The sum of the squared activities in a layer can b…

重塑基于相似性目标的Forward-Forward算法

Reshaping the Forward-Forward Algorithm with a Similarity-Based Objective

本研究将Forward-Forward算法与相似性学习整合，提出FAUST方法，消除推理时多次正向传播，在多个数据集上显著提升准确率，接近反向传播性能。

SharpZO：通过仅前向传播的混合锐度感知视觉语言模型提示微调

SharpZO: Hybrid Sharpness-Aware Vision Language Model Prompt Tuning via Forward-Only Passes

本文提出SharpZO，一种仅用前向传播的混合锐度感知零阶优化方法，通过锐度感知预热和两阶段优化，显著提升视觉语言模型微调的准确性与收敛速度。

A framework for the fine-grained evaluation of the instantaneous expected value of soccer possessions

The expected possession value (EPV) of a soccer possession represents the likelihood of a team scoring or receiving the next goal at any time instance. By decomposing the EPV into a series of subcomponents that are estimated separately, we develop a comprehensive analysis framework providing soccer practitioners with the ability to evaluate the impact of both observed and potential actions. We show we can obtain calibrated models for all the components of EPV, including a set of yet-unexplored problems in soccer. We produce visually-interpretable probability surfaces for potential passes from …

局部强化学习与基于动作的均方Q函数

Local Reinforcement Learning with Action-Conditioned Root Mean Squared Q-Functions

本文提出基于动作的均方Q函数，通过时间差分学习实现局部强化学习，优于现有无反向传播方法，在MinAtar和DeepMind控制套件中表现优异。

反向状态策略是学习算法的一部分

Backward-State Policy Is Part of the Learning Algorithm

本文论证反向传播中张量读取策略（反向状态策略）是学习算法的一部分，通过理论推导和实验证明复制精度与最终损失无法判断其正确性，需逐次使用检查。

通过计算图上的前向-后向算法估计多类动态出行生成需求

Estimating multi-class dynamic origin-destination demand through a forward-backward algorithm on computational graphs

本文提出了一种基于计算图的多类动态出行生成需求估计框架（MCDODE），通过前向-后向算法和树基累积曲线估计OD需求梯度，以解决大规模交通网络中多类时空车流估计的挑战。

使用 Forward-Forward 算法训练的网络中的涌现表征

Emergent representations in networks trained with the Forward-Forward algorithm

本研究发现，使用 Forward-Forward 算法训练的网络能自发形成类似大脑皮层的高稀疏性类别特定表征，且该现象也可在采用相同目标的 Backpropagation 训练中出现。

Semi-Supervised Natural Language Approach for Fine-Grained Classification of Medical Reports

Although machine learning has become a powerful tool to augment doctors in clinical analysis, the immense amount of labeled data that is necessary to train supervised learning approaches burdens each development task as time and resource intensive. The vast majority of dense clinical information is stored in written reports, detailing pertinent patient information. The challenge with utilizing natural language data for standard model development is due to the complex nature of the modality. In this research, a model pipeline was developed to utilize an unsupervised approach to train an encoder…

SeFAR：基于时序扰动与学习稳定化的半监督细粒度动作识别

SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization

本文提出半监督细粒度动作识别框架SeFAR，通过双层时序表示、时序扰动增强和自适应调节稳定学习，在多个细粒度与粗粒度数据集上取得最优性能。

MEMO：面向超长上下文大语言模型训练的细粒度张量管理

MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training

针对超长上下文LLM训练中GPU显存不足、碎片化严重及MFU低的问题，提出MEMO框架，通过细粒度激活卸载、逐令牌重计算与双层MIP显存复用优化，实现高MFU的百万级长序列7B模型训练。

AREAL-DTA：用于大语言模型高效强化学习的动态树注意力机制

AREAL-DTA: Dynamic Tree Attention for Efficient Reinforcement Learning of Large Language Models

针对RL后训练中rollout序列共享前缀导致计算冗余的问题，提出基于深度优先搜索的动态树注意力机制，结合负载均衡分布式批处理，实现高效前缀共享，训练吞吐量提升最高8.31倍。

基于视觉Transformer的开源两阶段细粒度车辆分类流水线

An Open-Source Two-Stage Computer Vision Pipeline for Fine-Grained Vehicle Classification using Vision Transformers

提出一个结合RT-DETR检测器和微调ViT-Base/16的两阶段流水线，用于六类车身分类，并引入置信度弃权机制，在分布内和分布外数据集上分别达到0.94和0.89的准确率。