Non-line-of-sight sensing of human activities in complex environments is enabled by multiple-input multiple-output through-the-wall radar (TWR). However, the distinctiveness of micro-Doppler signature between similar indoor human activities such as gun carrying and normal walking is minimal, while the large scale of input images required for effective identification utilizing time-frequency spectrograms creates challenges for model training and inference efficiency. To address this issue, the Chebyshev-time map is proposed in this paper, which is a method characterizing micro-Doppler signature using polynomial orders. The parametric kinematic models for human motion and the TWR echo model are first established. Then, a time-frequency feature representation method based on orthogonal Chebyshev polynomial decomposition is proposed. The kinematic envelopes of the torso and limbs are extracted, and the time-frequency spectrum slices are mapped into a robust Chebyshev-time coefficient space, preserving the multi-order morphological detail information of time-frequency spectrum. Numerical simulations and experiments are conducted to verify the effectiveness of the proposed method, which demonstrates the capability to characterize armed and unarmed indoor human activities while effectively compressing the scale of the time-frequency spectrum to achieve a balance between recognition accuracy and input data dimensions. The open-source code of this paper can be found in: https://github.com/JoeyBGOfficial/Represent-Micro-Doppler-Signature-in-Orders.
Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所）
Unique micro-Doppler signature ($\boldsymbolμ$-D) of a human body motion can be analyzed as the superposition of different body parts $\boldsymbolμ$-D signatures. Extraction of human limbs $\boldsymbolμ$-D signatures in real-time can be used to detect, classify and track human motion especially for safety application. In this paper, two methods are combined to simulate $\boldsymbolμ$-D signatures of a walking human. Furthermore, a novel limbs $μ$-D signature time independent decomposition feasibility study is presented based on features as $μ$-D signatures and range profiles also known as micr…
The identification of pedestrians using radar micro-Doppler signatures has become a hot topic in recent years. In this paper, we propose a multi-characteristic learning (MCL) model with clusters to jointly learn discrepant pedestrian micro-Doppler signatures and fuse the knowledge learned from each cluster into final decisions. Time-Doppler spectrogram (TDS) and signal statistical features extracted from FMCW radar, as two categories of micro-Doppler signatures, are used in MCL to learn the micro-motion information inside pedestrians' free walking patterns. The experimental results show that o…
Micro-Doppler signature is a potent feature that has been used for target identification and micro-motion parameter estimation. The extraction of high frequency micro-Doppler signature from frequency modulated continuous wave (FMCW) radar along with the target range and velocity is the problem considered in this article. The severe aliasing of the high micro-Doppler frequency spread is circumvented by the fast time processing in the proposed method. The use of range-Doppler (RD) filtering and empirical mode decomposition (EMD) enables effective out-of-band and in-band noise suppression. Simula…
Motion tracking systems based on optical sensors typically often suffer from issues, such as poor lighting conditions, occlusion, limited coverage, and may raise privacy concerns. More recently, radio frequency (RF)-based approaches using commercial WiFi devices have emerged which offer low-cost ubiquitous sensing whilst preserving privacy. However, the output of an RF sensing system, such as Range-Doppler spectrograms, cannot represent human motion intuitively and usually requires further processing. In this study, MDPose, a novel framework for human skeletal motion reconstruction based on Wi…
Through-the-Wall Radar Human Activity Micro-Doppler Signature Representation Method Based on Joint Boulic-Sinusoidal Pendulum Model
School of Information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）；Jiaxing Research Center of Beijing Institute of Technology（北京理工大学嘉兴研究院）；Key Laboratory of Electronic and Information Technology in Satellite Navigation, Beijing Institute of Technology（北京理工大学卫星导航电子信息技术重点实验室）
We introduce a simple but effective technique in automatic hand gesture recognition using radar. The proposed technique classifies hand gestures based on the envelopes of their micro-Doppler signatures. These envelopes capture the distinctions among different hand movements and their corresponding positive and negative Doppler frequencies which are generated during each gesture act. We detect the positive and negative envelopes separately, and form a feature vector of their augmentation. We use the $k$-nearest neighbor ($k$NN) classifier and Manhattan distance (L1) measure, in lieu of Euclidea…
In considering human-machine interface (HMI) for smart environment, a simple but effective method is proposed for automatic arm motion recognition with a Doppler radar sensor. Arms, in lieu of hands, have stronger radar cross-section and can be recognized from relatively longer distances. An energy-based thresholding algorithm is applied to the spectrograms to extract the micro-Doppler (MD) signature envelopes. The positive and negative frequency envelopes are concatenated to form a feature vector. The nearest neighbor (NN) classifier with Manhattan distance (L1) is then used to recognize the …
Hand and arm gesture recognition using the radio frequency (RF) sensing modality proves valuable in manmachine interface and smart environment. In this paper, we use curve matching techniques for measuring the similarity of the maximum instantaneous Doppler frequencies corresponding to different arm gestures. In particular, we apply both Frechet and dynamic time warping (DTW) distances that, unlike the Euclidean (L2) and Manhattan (L1) distances, take into account both the location and the order of the points for rendering two curves similar or dissimilar. It is shown that improved arm gesture…
Penetrating detection and recognition of behind wall indoor human activities has drawn great attentions from social security and emergency service department in recent years since intelligent surveillance aforehand could avail the proper decision making before operations being carried out. However, due to the influence of the wall effects, the obtained micro-Doppler signatures would be severely degenerated by strong near zero-frequency DC components, which would inevitably smear the detailed characteristic features of different behind wall motions in time-frequency (TF) map and further hinder …
Objective: In this paper, we demonstrate the applicability of radar for gait classification with application to home security, medical diagnosis, rehabilitation and assisted living. Aiming at identifying changes in gait patterns based on radar micro-Doppler signatures, this work is concerned with solving the intra motion category classification problem of gait recognition. Methods: New gait classification approaches utilizing physical features, subspace features and sum-of-harmonics modeling are presented and their performances are evaluated using experimental K-band radar data of four test su…
Siemens Corporate Technology（西门子企业技术部）；Center for Advanced Communications, Villanova University（维拉诺瓦大学高级通信中心）；University of Alabama（阿拉巴马大学）
Deep neural networks (DNNs) have recently received vast attention in applications requiring classification of radar returns, including radar-based human activity recognition for security, smart homes, assisted living, and biomedicine. However,acquiring a sufficiently large training dataset remains a daunting task due to the high human costs and resources required for radar data collection. In this paper, an extended approach to adversarial learning is proposed for generation of synthetic radar micro-Doppler signatures that are well-adapted to different environments. The synthetic data is evalu…
PSL University（巴黎文理研究大学）；Mines Paris（巴黎国立高等矿业学院）；Thales LAS France（法国泰雷兹机载系统公司）
Near out-of-distribution detection (OODD) aims at discriminating semantically similar data points without the supervision required for classification. This paper puts forward an OODD use case for radar targets detection extensible to other kinds of sensors and detection scenarios. We emphasize the relevance of OODD and its specific supervision requirements for the detection of a multimodal, diverse targets class among other similar radar targets and clutter in real-life critical systems. We propose a comparison of deep and non-deep OODD methods on simulated low-resolution pulse radar micro-Dop…
Joint communication and sensing (JCAS) technology has been regarded as one of the innovations in the 6G network. With the channel modeling proposed by the 3rd Generation Partnership Project (3GPP) TR 38.901, this paper investigates the sensing capability using the millimeter-wave (mmWave) band with an orthogonal frequency division multiplexing (OFDM) waveform. Based on micro-Doppler (MD) analysis, we present two case studies, i.e., fan speed detection and human activity recognition, to demonstrate the target modeling with micro-motions, backscattering signal construction, and MD signature extr…
Recently, deep neural networks (DNNs) have been the subject of intense research for the classification of radio frequency (RF) signals, such as synthetic aperture radar (SAR) imagery or micro-Doppler signatures. However, a fundamental challenge is the typically small amount of data available due to the high costs and resources required for measurements. Small datasets limit the depth of DNNs implementable, and limit performance. In this work, a novel method for generating diversified radar micro-Doppler signatures using Kinect-based motion capture simulations is proposed as a training database…
Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所）
Radar sensors can be used for analyzing the induced frequency shifts due to micro-motions in both range and velocity dimensions identified as micro-Doppler ($\boldsymbolμ$-D) and micro-Range ($\boldsymbolμ$-R), respectively. Different moving targets will have unique $\boldsymbolμ$-D and $\boldsymbolμ$-R signatures that can be used for target classification. Such classification can be used in numerous fields, such as gait recognition, safety and surveillance. In this paper, a 25 GHz FMCW Single-Input Single-Output (SISO) radar is used in industrial safety for real-time human-robot identificatio…
In this work, we investigate the use of backscattered mm-wave radio signals for the joint tracking and recognition of identities of humans as they move within indoor environments. We build a system that effectively works with multiple persons concurrently sharing and freely moving within the same indoor space. This leads to a complicated setting, which requires one to deal with the randomness and complexity of the resulting (composite) backscattered signal. The proposed system combines several processing steps: at first, the signal is filtered to remove artifacts, reflections and random noise …
Micro-Doppler analysis has become increasingly popular in recent years owning to the ability of the technique to enhance classification strategies. Applications include recognising everyday human activities, distinguishing drone from birds, and identifying different types of vehicles. However, noisy time-frequency spectrograms can significantly affect the performance of the classifier and must be tackled using appropriate denoising algorithms. In recent years, deep learning algorithms have spawned many deep neural network-based denoising algorithms. For these methods, noise modelling is the mo…
Micro-Doppler signatures contain considerable information about target dynamics. However, the radar sensing systems are easily affected by noisy surroundings, resulting in uninterpretable motion patterns on the micro-Doppler spectrogram. Meanwhile, radar returns often suffer from multipath, clutter and interference. These issues lead to difficulty in, for example motion feature extraction, activity classification using micro Doppler signatures ($μ$-DS), etc. In this paper, we propose a latent feature-wise mapping strategy, called Feature Mapping Network (FMNet), to transform measured spectrogr…
Generalizable Indoor Human Activity Recognition Method Based on Micro-Doppler Corner Point Cloud and Dynamic Graph Learning
School of Information and Electronics, Beijing Institute of Technology（北京理工大学信息与电子学院）；Jiaxing Research Center of Beijing Institute of Technology（北京理工大学嘉兴研究院）；Key Laboratory of Electronic and Information Technology in Satellite Navigation, Beijing Institute of Technology（北京理工大学卫星导航电子信息技术重点实验室）
Micro-vibration, a ubiquitous nature phenomenon, can be seen as a characteristic feature on the objects, these vibrations always have tiny amplitudes which are much less than the wavelengths of the sensing systems, thus these motions information can only be reflected in the phase item of echo. Normally the conventional radar system can detect these micro vibrations through the time frequency analyzing, but these vibration characteristics can only be reflected by time-frequency spectrum, the spatial distribution of these micro vibrations can not be reconstructed precisely. Ghost imaging (GI), a…
This work presents an application of Integrated sensing and communication (ISAC) system for monitoring human activities directly related to healthcare. Real-time monitoring of humans can assist professionals in providing healthy living enabling technologies to ensure the health, safety, and well-being of people of all age groups. To enhance the human activity recognition performance of the ISAC system, we propose to use synthetic data generated through our human micro-Doppler simulator, SimHumalator to augment our limited measurement data. We generate a more realistic micro-Doppler signature d…
This paper considers human activity classification for an indoor radar system. Human motions generate nonstationary radar returns which represent Doppler and micro-Doppler signals. The time-frequency (TF) analysis of micro-Doppler signals can discern subtle variations on the motion by precisely revealing velocity components of various moving body parts. We consider radar for activity monitoring using TF-based machine learning approach exploiting both temporal and spatial degrees of freedom. The proposed approach captures different human motion representations more vividly in joint-variable dat…
A well established method to detect and classify human movements using Millimeter-Wave ( mmWave) devices is the time-frequency analysis of the small-scale Doppler effect (termed micro-Doppler) of the different body parts, which requires a regularly spaced and dense sampling of the Channel Impulse Response ( CIR). This is currently done in the literature either using special-purpose radar sensors, or interrupting communications to transmit dedicated sensing waveforms, entailing high overhead and channel utilization. In this work we present SPARCS, an integrated human sensing and communication s…
A recent paper (Neural Networks, {\bf 132} (2020), 253-268) introduces a straightforward and simple kernel based approximation for manifold learning that does not require the knowledge of anything about the manifold, except for its dimension. In this paper, we examine how the pointwise error in approximation using least squares optimization based on similarly localized kernels depends upon the data characteristics and deteriorates as one goes away from the training data. The theory is presented with an abstract localized kernel, which can utilize any prior knowledge about the data being locate…
A problem with synthetic aperture radar (SAR) is that, due to the poor penetrating action of electromagnetic waves inside solid bodies, the capability to observe inside distributed targets is precluded. Under these conditions, imaging action is provided only on the surface of distributed targets. The present work describes an imaging method based on the analysis of micro-movements on the Khnum-Khufu Pyramid, which are usually generated by background seismic waves. The results obtained prove to be very promising, as high-resolution full 3D tomographic imaging of the pyramid's interior and subsu…
The applicability of Doppler radar for gait analysis is investigated by quantitatively comparing the measured biomechanical parameters to those obtained using motion capturing and ground reaction forces. Nineteen individuals walked on a treadmill at two different speeds, where a radar system was positioned in front of or behind the subject. The right knee angle was confined by an adjustable orthosis in five different degrees. Eleven gait parameters are extracted from radar micro-Doppler signatures. Here, new methods for obtaining the velocities of individual lower limb joints are proposed. Fur…
