Neuromorphic vision sensor is a new bio-inspired imaging paradigm that reports asynchronous, continuously per-pixel brightness changes called `events' with high temporal resolution and high dynamic range. So far, the event-based image reconstruction methods are based on artificial neural networks (ANN) or hand-crafted spatiotemporal smoothing techniques. In this paper, we first implement the image reconstruction work via fully spiking neural network (SNN) architecture. As the bio-inspired neural networks, SNNs operating with asynchronous binary spikes distributed over time, can potentially lead to greater computational efficiency on event-driven hardware. We propose a novel Event-based Video reconstruction framework based on a fully Spiking Neural Network (EVSNN), which utilizes Leaky-Integrate-and-Fire (LIF) neuron and Membrane Potential (MP) neuron. We find that the spiking neurons have the potential to store useful temporal information (memory) to complete such time-dependent tasks. Furthermore, to better utilize the temporal information, we propose a hybrid potential-assisted framework (PA-EVSNN) using the membrane potential of spiking neuron. The proposed neuron is referred as Adaptive Membrane Potential (AMP) neuron, which adaptively updates the membrane potential according to the input spikes. The experimental results demonstrate that our models achieve comparable performance to ANN-based models on IJRR, MVSEC, and HQF datasets. The energy consumptions of EVSNN and PA-EVSNN are 19.36$\times$ and 7.75$\times$ more computationally efficient than their ANN architectures, respectively.
Pacific Northwest National Laboratory（太平洋西北国家实验室）；Drexel University（德雷塞尔大学）；National Research Council of Canada（加拿大国家研究委员会）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）
The Locally Competitive Algorithm (LCA) uses local competition between non-spiking leaky integrator neurons to infer sparse representations, allowing for potentially real-time execution on massively parallel neuromorphic architectures such as Intel's Loihi processor. Here, we focus on the problem of inferring sparse representations from streaming video using dictionaries of spatiotemporal features optimized in an unsupervised manner for sparse reconstruction. Non-spiking LCA has previously been used to achieve unsupervised learning of spatiotemporal dictionaries composed of convolutional kerne…
SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction
The Hamlyn Centre for Robotic Surgery, Imperial College London（伦敦帝国理工学院哈姆林 robotic外科中心）；Hawkes Institute, University College London（伦敦大学学院霍克斯研究所）；University of Liverpool（利物浦大学）
School of Computer and Artificial Intelligence, Zhengzhou University（计算机与人工智能学院，郑州大学）；Wuhan University（武汉大学）；Harbin Institution of Technology（哈尔滨理工大学）；Dalian Maritime University（大连海事大学）；National Tsinghua University（清华大学）
Xidian University（西安电子科技大学）；Shandong Normal University（山东师范大学）；Northwestern Polytechnical University（西北工业大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Single image dehazing as a fundamental low-level vision task, is essential for the development of robust intelligent surveillance system. In this paper, we make an early effort to consider dehazing robustness under variational haze density, which is a realistic while under-studied problem in the research filed of singe image dehazing. To properly address this problem, we propose a novel density-variational learning framework to improve the robustness of the image dehzing model assisted by a variety of negative hazy images, to better deal with various complex hazy scenarios. Specifically, the d…
Manycore Tech Inc.（Manycore科技公司）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of California, Berkeley（加州大学伯克利分校）；Zhejiang University（浙江大学）
In this paper, we revisit the long-standing problem of automatic reconstruction of 3D objects from single line drawings. Previous optimization-based methods can generate compact and accurate 3D models, but their success rates depend heavily on the ability to (i) identifying a sufficient set of true geometric constraints, and (ii) choosing a good initial value for the numerical optimization. In view of these challenges, we propose to train deep neural networks to detect pairwise relationships among geometric entities (i.e., edges) in the 3D object, and to predict initial depth value of the vert…
University of Electronic Science and Technology of China（电子科技大学）；National University of Singapore（新加坡国立大学）；National University of Defense Technology（国防科技大学）；University of Science and Technology of China（中国科学技术大学）；CASIA（中国科学院自动化研究所）
3D face reconstruction from a single 2D image is a challenging problem with broad applications. Recent methods typically aim to learn a CNN-based 3D face model that regresses coefficients of 3D Morphable Model (3DMM) from 2D images to render 3D face reconstruction or dense face alignment. However, the shortage of training data with 3D annotations considerably limits performance of those methods. To alleviate this issue, we propose a novel 2D-assisted self-supervised learning (2DASL) method that can effectively use "in-the-wild" 2D face images with noisy landmark information to substantially im…
Although recent deep learning methods, especially generative models, have shown good performance in fast magnetic resonance imaging, there is still much room for improvement in high-dimensional generation. Considering that internal dimensions in score-based generative models have a critical impact on estimating the gradient of the data distribution, we present a new idea, low-rank tensor assisted k-space generative model (LR-KGM), for parallel imaging reconstruction. This means that we transform original prior information into high-dimensional prior information for learning. More specifically,…
Spiking Variational Graph Representation Inference for Video Summarization
Department of Computer Science and Technology, Harbin Institute of Technology（计算机科学与技术系，哈尔滨工业大学）；Harbin Institute of Technology Suzhou Research Institute（哈尔滨工业大学苏州研究 institute）；School of mathmatical Sciences/Multi-Hazard Early Warning Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China（数学科学学院/四川省多灾种早期预警重点实验室，电子科技大学）；School of Electronic Engineering and Computer Science, Institute of Digital Media, Peking University（电子工程与计算机科学系，数字媒体研究院，北京大学）
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Department of Mechanical Engineering, Carnegie Mellon University（卡内基梅隆大学机械工程系）
This work presents dense stereo reconstruction using high-resolution images for infrastructure inspections. The state-of-the-art stereo reconstruction methods, both learning and non-learning ones, consume too much computational resource on high-resolution data. Recent learning-based methods achieve top ranks on most benchmarks. However, they suffer from the generalization issue due to lack of task-specific training data. We propose to use a less resource demanding non-learning method, guided by a learning-based model, to handle high-resolution images and achieve accurate stereo reconstruction.…
RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；School of Computer Science and Technology, Harbin University of Science and Technology（哈尔滨理工大学计算机科学与技术学院）
Single-Image HDR Reconstruction Assisted Ghost Suppression and Detail Preservation Network for Multi-Exposure HDR Imaging
Faculty of Information Engineering and Automation, Kunming University of Science and Technology（昆明理工大学信息工程与自动化学院）；School of Information Science and Engineering, Yunnan University（云南大学信息科学与工程学院）
Shanghai Jiao Tong University（上海交通大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能科技有限公司）；ShanghaiTech University（上海科技大学）
In clinical practice, multi-modal magnetic resonance imaging (MRI) with different contrasts is usually acquired in a single study to assess different properties of the same region of interest in the human body. The whole acquisition process can be accelerated by having one or more modalities under-sampled in the $k$-space. Recent research has shown that, considering the redundancy between different modalities, a target MRI modality under-sampled in the $k$-space can be more efficiently reconstructed with a fully-sampled reference MRI modality. However, we find that the performance of the afore…
GA-GS: Generation-Assisted Gaussian Splatting for Static Scene Reconstruction
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；School of Artificial Intelligence and Data Science, University of Science and Technology of China（中国科学技术大学人工智能与数据科学学院）；The Chinese University of Hong Kong（香港中文大学）；Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；School of information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）
Acquiring Submillimeter-Accurate Multi-Task Vision Datasets for Computer-Assisted Orthopedic Surgery
University Hospital Balgrist, University of Zurich（苏黎世大学医院巴尔格里斯分院、苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Institut de Recherche en Informatique de Toulouse（图卢兹计算机科学研究所）
SpikingNeRF: Making Bio-inspired Neural Networks See through the Real World
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；China Electric Power Research Institute Co., Ltd（中国电力科学研究院有限公司）
Spatial-Related Sensors Matters: 3D Human Motion Reconstruction Assisted with Textual Semantics
Beijing Advanced Innovation Center for Materials Genome Engineering（北京材料基因组工程先进创新中心）；University of Science and Technology Beijing（北京科技大学）；Key Laboratory of Intelligent Bionic Unmanned Systems, Ministry of Education（智能仿生无人系统教育部重点实验室）；Institute of Materials Intelligent Technology, Liaoning Academy of Materials（辽宁材料实验室材料智能技术研究所）
QuantEIT: Ultra-Lightweight Quantum-Assisted Inference for Chest Electrical Impedance Tomography
SMART Group, Institute for Imaging, Data and Communications (IDCOM), School of Engineering, The University of Edinburgh（SMART集团，成像、数据与通讯研究所（IDCOM），工程学院，爱丁堡大学）；State Key Laboratory of Complex Severe and Rare Diseases, Department of Critical Care Medicine, Peking Union Medical College, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences（复杂严重罕见病国家重点实验室，重症医学科，北京友谊医院，中国医学科学院）
Computer vision and robotics are being increasingly applied in medical interventions. Especially in interventions where extreme precision is required they could make a difference. One such application is robot-assisted retinal microsurgery. In recent works, such interventions are conducted under a stereo-microscope, and with a robot-controlled surgical tool. The complementarity of computer vision and robotics has however not yet been fully exploited. In order to improve the robot control we are interested in 3D reconstruction of the anatomy and in automatic tool localization using a stereo mic…
Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction
Mahindra University（马欣德拉大学）；MU-VT Interdisciplinary Advanced Research Centre for Transformative Technologies, Mahindra University（马欣德拉大学MU-VT变革性技术跨学科高级研究中心）
JADE-GS: Joint Allocation of Deblurring Evidence for Event-Assisted 3D Gaussian Splatting
School of Mechatronic Engineering and Automation, Shanghai University（上海大学机电工程与自动化学院）；Shanghai Baoshan Shangda General Intelligent Robotics Research Institute（上海宝山尚大通用智能机器人研究院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）
Nokia Bell Labs（诺基亚贝尔实验室）；Robert Bosch GmbH（罗伯特·博世有限公司）
To support Industry 4.0 applications with haptics and human-machine interaction, 6G requires a new framework that is fully autonomous, visual, and interactive. In this paper, we provide an end-to-end solution, HARU, for private network planning services, especially industrial networks. The solution consists of the following functions: collecting visual and sensory data from the user device, reconstructing 3D radio propagation environment and conducting network planning on a server, and visualizing network performance with AR on the user device with enabled haptic feedback. The functions are em…
