NSFlow：用于视觉里程计的端到端可微分神经符号光流
NSFlow: End-to-End Differentiable Neuro-Symbolic Optical Flow for Visual Odometry
浏览论文内容
中文总结 AI 辅助
提出NSFlow混合神经符号光流框架，结合CNN特征提取与可微分LK优化，实现端到端训练，在VIO中平均误差降低42%，实时运行于嵌入式平台。
中文摘要 AI 辅助
稀疏光流提供稳定的帧间对应关系，在视觉里程计（VO）和视觉惯性里程计（VIO）中发挥着关键作用。经典的基于优化的方法，如Lucas-Kanade（LK），在小位移情况下表现良好，但对大运动和光照变化敏感。现代的基于回归的学习方法虽然在复杂场景中更为鲁棒，但通常计算量大且缺乏显式的几何一致性，使其不太适合高效的VO/VIO前端。为弥合这一差距，我们提出了一种混合神经符号框架，结合了两种范式的优势。我们的方法使用卷积神经网络（CNN）提取鲁棒的特征表示，并将其输入到可微分的LK优化器中，以端到端可训练的方式估计光流。通过隐式微分，梯度在迭代求解器中传播，实现特征提取和光流估计的联合优化。所得系统无缝集成到现有的VO/VIO流程中，并在嵌入式平台上实时运行。实验表明，我们的方法在动态光照和低纹理等挑战性条件下优于传统的基于优化的光流，同时比纯回归方法具有更高的准确性和更低的延迟。当部署在VIO系统中时，我们的方法显示出显著的性能提升，在具有挑战性的数据集上平均误差降低了42%，同时增强了跟踪稳定性。代码已公开。
英文摘要
Sparse optical flow provides stable inter-frame correspondence, playing a key role in Visual Odometry (VO) and Visual-Inertial Odometry (VIO). Classical optimization-based methods, such as Lucas-Kanade (LK), perform well under small displacements but are sensitive to large motions and illumination changes. Modern regression-based learning methods, while more robust in complex scenes, are often computationally heavy and lack explicit geometric consistency, making them less suitable for efficient VO/VIO front-ends. To bridge this gap, we propose a hybrid neuro-symbolic framework that combines the strengths of both paradigms. Our method uses a Convolutional Neural Network (CNN) to extract robust feature representations, which is fed into a differentiable LK optimizer to estimate optical flow in an end-to-end trainable manner. Through implicit differentiation, gradients are propagated across the iterative solver, enabling joint optimization of feature extraction and flow estimation. The resulting system integrates seamlessly into existing VO/VIO pipelines and runs in real-time on embedded platforms. Experiments show that our method outperforms conventional optimization-based flow in challenging conditions such as dynamic lighting and low texture, while also achieving higher accuracy and lower latency than purely regression-based alternatives. When deployed in a VIO system, our method demonstrates significant performance improvement, achieving an average error reduction of 42\% on challenging datasets while enhancing tracking stability. The code is publicly available.
发表机构
- State Key Laboratory of Intelligent Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology（华中科技大学机械科学与工程学院智能制造装备与技术国家重点实验室）
机构由 AI 辅助整理，请以论文原文为准。
