arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.08442cs.CVcs.AIcs.LG
收藏

深度平衡光流估计

Deep Equilibrium Optical Flow Estimation

Carnegie Mellon University(卡内基梅隆大学);Peking University(北京大学);Bosch Center for AI(博世人工智能中心)

Shaojie Bai, Zhengyang Geng, Yash Savani, J. Zico Kolter

摘要:

许多最新的国家的最先进的(SOTA)光流模型使用有限步的经常性更新操作,以模拟传统的算法,通过鼓励迭代细化到一个稳定的流量估计。然而,这些RNN施加了大量的计算和内存开销,并且没有直接训练来对这种稳定的估计进行建模。它们可能收敛得很差,从而导致性能下降。为了克服这些缺点,我们提出了深度平衡(DEQ)流估计,一种直接解决流作为隐式层的无限级固定点(使用任何黑盒求解器)的方法,并通过该固定点进行分析(因此需要$O(1)$训练内存)。这种隐式深度方法不基于任何特定模型,因此可以应用于广泛的SOTA流量估计模型设计。这些DEQ流量估计器的使用允许我们更快地计算流量,例如,固定点重用和不精确的梯度,消耗比循环对应物少$4\sim6\times$倍的训练存储器,并且在相同的计算预算下实现更好的结果。此外,我们提出了一种新型的稀疏定点校正方案来稳定我们的DEQ流量估计器,这解决了DEQ模型长期面临的挑战。我们在各种现实环境中测试我们的方法,并表明它改善了Sintel和KITTI数据集上的SOTA方法,具有更好的计算和内存效率。

英文摘要:

Many recent state-of-the-art (SOTA) optical flow models use finite-step recurrent update operations to emulate traditional algorithms by encouraging iterative refinements toward a stable flow estimation. However, these RNNs impose large computation and memory overheads, and are not directly trained to model such stable estimation. They can converge poorly and thereby suffer from performance degradation. To combat these drawbacks, we propose deep equilibrium (DEQ) flow estimators, an approach that directly solves for the flow as the infinite-level fixed point of an implicit layer (using any black-box solver), and differentiates through this fixed point analytically (thus requiring $O(1)$ training memory). This implicit-depth approach is not predicated on any specific model, and thus can be applied to a wide range of SOTA flow estimation model designs. The use of these DEQ flow estimators allows us to compute the flow faster using, e.g., fixed-point reuse and inexact gradients, consumes $4\sim6\times$ times less training memory than the recurrent counterpart, and achieves better results with the same computation budget. In addition, we propose a novel, sparse fixed-point correction scheme to stabilize our DEQ flow estimators, which addresses a longstanding challenge for DEQ models in general. We test our approach in various realistic settings and show that it improves SOTA methods on Sintel and KITTI datasets with substantially better computational and memory efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑