首页/论文/2001.03455

arXiv 2001.034552020-01cs.CV

A Differentiable Recurrent Surface for Asynchronous Event-Based Data

Politecnico di Milano（米兰理工大学）

Marco Cannici, Marco Ciccone, Andrea Romanoni, Matteo Matteucci

备注：: 23 pages, 6 figures

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

Dynamic Vision Sensors (DVSs) asynchronously stream events in correspondence of pixels subject to brightness changes. Differently from classic vision devices, they produce a sparse representation of the scene. Therefore, to apply standard computer vision algorithms, events need to be integrated into a frame or event-surface. This is usually attained through hand-crafted grids that reconstruct the frame using ad-hoc heuristics. In this paper, we propose Matrix-LSTM, a grid of Long Short-Term Memory (LSTM) cells that efficiently process events and learn end-to-end task-dependent event-surfaces. Compared to existing reconstruction approaches, our learned event-surface shows good flexibility and expressiveness on optical flow estimation on the MVSEC benchmark and it improves the state-of-the-art of event-based object classification on the N-Cars dataset.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction

Robotics and Perception Group, University of Zurich（苏黎世大学机器人与感知组）

Event cameras are novel vision sensors that report per-pixel brightness changes as a stream of asynchronous "events". They offer significant advantages compared to standard cameras due to their high temporal resolution, high dynamic range and lack of motion blur. However, events only measure the varying component of the visual signal, which limits their ability to encode scene context. By contrast, standard cameras measure absolute intensity frames, which capture a much richer representation of the scene. Both sensors are thus complementary. However, due to the asynchronous nature of events, c…

2021-02-18cs.CV

Arcee：基于Mamba SSMs的生成式视觉建模可微分循环状态链

Arcee: Differentiable Recurrent State Chain for Generative Vision Modeling with Mamba SSMs

New Jersey Institute of Technology（新泽西理工学院）；University of California, Riverside（加州大学河滨分校）；McGill University（麦吉尔大学）

针对Mamba SSMs在视觉建模中跨块丢弃终端状态空间表示的问题，提出Arcee跨块循环状态链，复用前块终端SSR作为后块初始条件，在CelebA-HQ上使FID降低5.4倍，兼容现有变体且无额外参数。

2025-11-17cs.CV

缓存Transformer：利用可微分记忆缓存改进Transformer

Cached Transformers: Improving Transformers with Differentiable Memory Cache

The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；Tencent Inc（腾讯公司）

提出Cached Transformer，采用门控循环缓存注意力，扩展自注意力以处理长距离依赖，在六项语言和视觉任务中取得显著进展，并优于现有记忆方法。

2023-12-20cs.CV

End-to-End Learning of Representations for Asynchronous Event-Based Data

University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Ryerson University（瑞尔森大学）；Samsung AI Centre Toronto（三星多伦多人工智能中心）

Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events". They have appealing advantages over frame-based cameras for computer vision, including high temporal resolution, high dynamic range, and no motion blur. Due to the sparse, non-uniform spatiotemporal layout of the event signal, pattern recognition algorithms typically aggregate events into a grid-based representation and subsequently process it by a standard vision pipeline, e.g., Convolutional Neural Network (CNN). In this work, we introduce a general framework to convert…

2019-08-20cs.CV

Differentiable Event Stream Simulator for Non-Rigid 3D Tracking

Saarland University（萨尔兰大学）；MPI for Informatics, SIC（马克斯·普朗克信息学研究所，SIC）；Google Inc.（谷歌公司）

This paper introduces the first differentiable simulator of event streams, i.e., streams of asynchronous brightness change signals recorded by event cameras. Our differentiable simulator enables non-rigid 3D tracking of deformable objects (such as human hands, isometric surfaces and general watertight meshes) from event streams by leveraging an analysis-by-synthesis principle. So far, event-based tracking and reconstruction of non-rigid objects in 3D, like hands and body, has been either tackled using explicit event trajectories or large-scale datasets. In contrast, our method does not require…

2021-04-30cs.CV

DDRprog: A CLEVR Differentiable Dynamic Reasoning Programmer

Stanford University（斯坦福大学）

We present a novel Dynamic Differentiable Reasoning (DDR) framework for jointly learning branching programs and the functions composing them; this resolves a significant nondifferentiability inhibiting recent dynamic architectures. We apply our framework to two settings in two highly compact and data efficient architectures: DDRprog for CLEVR Visual Question Answering and DDRstack for reverse Polish notation expression evaluation. DDRprog uses a recurrent controller to jointly predict and execute modular neural programs that directly correspond to the underlying question logic; it explicitly f…

2018-03-30cs.CV

更多相关论文收起更多论文

低延迟基于事件的对象检测与空间稀疏线性注意力

Low-Latency Event-Based Object Detection with Spatially-Sparse Linear Attention

Tsinghua University（清华大学）；University of Zurich（苏黎世大学）；ShanghaiTech University（上海交通大学）

本文提出SSLA，一种空间稀疏线性注意力机制，用于提升基于事件的目标检测效率与精度，实现低延迟性能。

2026-08-27cs.CV

Event-based Monocular Dense Depth Estimation with Recurrent Transformers

Event cameras, offering high temporal resolutions and high dynamic ranges, have brought a new perspective to address common challenges (e.g., motion blur and low light) in monocular depth estimation. However, how to effectively exploit the sparse spatial information and rich temporal cues from asynchronous events remains a challenging endeavor. To this end, we propose a novel event-based monocular depth estimator with recurrent transformers, namely EReFormer, which is the first pure transformer with a recursive mechanism to process continuous event streams. Technically, for spatial modeling, a…

2022-12-06cs.CV

Recurrent Pixel Embedding for Instance Grouping

University of California, Irvine（加州大学欧文分校）

We introduce a differentiable, end-to-end trainable framework for solving pixel-level grouping problems such as instance segmentation consisting of two novel components. First, we regress pixels into a hyper-spherical embedding space so that pixels from the same group have high cosine similarity while those from different groups have similarity below a specified margin. We analyze the choice of embedding dimension and margin, relating them to theoretical results on the problem of distributing points uniformly on the sphere. Second, to group instances, we utilize a variant of mean-shift cluster…

2017-12-22cs.CVcs.LGcs.MM

Recurrent 3D Attentional Networks for End-to-End Active Object Recognition

University of Maryland, College Park（马里兰大学帕克分校）；Shenzhen University（深圳大学）

Active vision is inherently attention-driven: The agent actively selects views to attend in order to fast achieve the vision task while improving its internal representation of the scene being observed. Inspired by the recent success of attention-based models in 2D vision tasks based on single RGB images, we propose to address the multi-view depth-based active object recognition using attention mechanism, through developing an end-to-end recurrent 3D attentional network. The architecture takes advantage of a recurrent neural network (RNN) to store and update an internal representation. Our mod…

2022-01-11cs.CV

StreamingFlow：基于神经常微分方程的异步多模态数据流流式占用预测

StreamingFlow: Streaming Occupancy Forecasting with Asynchronous Multi-modal Data Streams via Neural Ordinary Differential Equation

School of Vehicle and Mobility, Tsinghua University（清华大学车辆与学院）；KargoBot, Inc（卡戈机器人公司）；DiDi Chuxing（滴滴出行）

StreamingFlow 提出将神经常微分方程融入循环神经网络，实现异步多模态数据流下的任意时刻流式 BEV 占用预测，并在 nuScenes 和 Lyft L5 上显著超越现有方法。

2024-06-11cs.CV

GRVS：一种通用且递归的方法用于单目动态视图合成

GRVS: a Generalizable and Recurrent Approach to Monocular Dynamic View Synthesis

本文提出GRVS方法，通过递归循环和动态输入平面扫描，实现高精度的单目动态视图合成，优于现有多种基于高斯点划和扩散的方法。

2026-03-31cs.CV

结合事件与帧的深度视觉里程计

Deep Visual Odometry with Events and Frames

University of Zurich（苏黎世大学）；European Space Agency（欧洲空间局）

针对事件与图像模态难以异步融合的问题，提出端到端RAMP-VO及RAMP编码器，在仿真训练后于新旧基准上显著提升精度与推理速度。

2024-09-10cs.CV

一种用于基于事件的人体姿态估计的时序密集连接循环网络

A Temporal Densely Connected Recurrent Network for Event-based Human Pose Estimation

Hunan Normal University（湖南师范大学）；Zhejiang University of Technology（浙江工业大学）；City University of Hong Kong（香港城市大学）；Soochow University（苏州大学）

本文提出一种密集连接循环架构，通过建模时序与非时序几何一致性来恢复基于事件相机的人体姿态估计中缺失的身体部位，并构建了大规模多模态数据集验证其有效性。

2023-04-06cs.CV

自监督事件表示：迈向 SoC FPGA 上的精确实时感知

Self-Supervised Event Representations: Towards Accurate, Real-Time Perception on SoC FPGAs

AGH University of Krakow（克拉科夫AGH科技大学）

本文提出自监督 SSER 方法，用 GRU 异步逐像素编码事件时间戳与极性，并在 Gen1、1 Mpx 上提升检测精度，同时实现亚微秒、1–2 W 的 SoC FPGA 部署。

2025-05-12cs.CV

BlinkTrack：基于事件与图像实现80 FPS以上的特征跟踪

BlinkTrack: Feature Tracking over 80 FPS via Events and Images

State Key Lab of CAD&CG, Zhejiang University（CAD与CG国家重点实验室，浙江大学）

针对事件相机缺乏纹理信息导致特征跟踪误差累积的问题，提出融合事件与灰度图像的BlinkTrack框架，通过双分支可微卡尔曼滤波器实现异步数据关联融合，性能优于现有方法且帧率超80 FPS。

2025-07-31cs.CV

Recurrent Multi-view Alignment Network for Unsupervised Surface Registration

University of Science and Technology of China（中国科学技术大学）；City University of Hong Kong（香港城市大学）；Zhejiang University（浙江大学）

Learning non-rigid registration in an end-to-end manner is challenging due to the inherent high degrees of freedom and the lack of labeled training data. In this paper, we resolve these two challenges simultaneously. First, we propose to represent the non-rigid transformation with a point-wise combination of several rigid transformations. This representation not only makes the solution space well-constrained but also enables our method to be solved iteratively with a recurrent framework, which greatly reduces the difficulty of learning. Second, we introduce a differentiable loss function that …

2021-04-13cs.CVcs.GR

深度学习为何能提升视觉同步定位与地图构建？

Why does Deep Learning Improve Visual SLAM?

Robotics and Perception Group, Department of Informatics, University of Zurich（苏黎世大学信息学系机器人与感知组）

探讨深度学习提升视觉SLAM性能的原因，通过对照研究发现其成功依赖于二维数据关联和不确定性，而非循环架构，凸显基于学习的范式对设计相关组件的必要性。

2026-09-22cs.CVcs.RO