Resource-efficient Deep Neural Networks for Automotive Radar Interference Mitigation
Graz University of Technology（格拉茨理工大学）；Infineon Technologies Austria AG（英飞凌科技奥地利股份公司）
Johanna Rock, Wolfgang Roth, Mate Toth, Paul Meissner, Franz Pernkopf
备注：
15 pages; published in IEEE Journal of Selected Topics in Signal Processing, Special Issue on Recent Advances in Automotive Radar Signal Processing, Volume: 15, Issue: 4, June 2021. arXiv admin note: text overlap with arXiv:2011.12706
发表信息：
IEEE Journal of Selected Topics in Signal Processing, vol. 15, no. 4, pp. 927-940, June 2021
Radar sensors are crucial for environment perception of driver assistance systems as well as autonomous vehicles. With a rising number of radar sensors and the so far unregulated automotive radar frequency band, mutual interference is inevitable and must be dealt with. Algorithms and models operating on radar data are required to run the early processing steps on specialized radar sensor hardware. This specialized hardware typically has strict resource-constraints, i.e. a low memory capacity and low computational power. Convolutional Neural Network (CNN)-based approaches for denoising and interference mitigation yield promising results for radar processing in terms of performance. Regarding resource-constraints, however, CNNs typically exceed the hardware's capacities by far.
In this paper we investigate quantization techniques for CNN-based denoising and interference mitigation of radar signals. We analyze the quantization of (i) weights and (ii) activations of different CNN-based model architectures. This quantization results in reduced memory requirements for model storage and during inference. We compare models with fixed and learned bit-widths and contrast two different methodologies for training quantized CNNs, i.e. the straight-through gradient estimator and training distributions over discrete weights. We illustrate the importance of structurally small real-valued base models for quantization and show that learned bit-widths yield the smallest models. We achieve a memory reduction of around 80\% compared to the real-valued baseline. Due to practical reasons, however, we recommend the use of 8 bits for weights and activations, which results in models that require only 0.2 megabytes of memory.
Autonomous radar has been an integral part of advanced driver assistance systems due to its robustness to adverse weather and various lighting conditions. Conventional automotive radars use digital signal processing (DSP) algorithms to process raw data into sparse radar pins that do not provide information regarding the size and orientation of the objects. In this paper, we propose a deep-learning based algorithm for radar object detection. The algorithm takes in radar data in its raw tensor representation and places probabilistic oriented bounding boxes around the detected objects in bird's-e…
Vitalent Consulting；Beihang University（北京航空航天大学）；Chalmers University of Technology（查尔姆斯理工大学）；The University of Sydney（悉尼大学）
Automotive radar provides reliable environmental perception in all-weather conditions with affordable cost, but it hardly supplies semantic and geometry information due to the sparsity of radar detection points. With the development of automotive radar technologies in recent years, instance segmentation becomes possible by using automotive radar. Its data contain contexts such as radar cross section and micro-Doppler effects, and sometimes can provide detection when the field of view is obscured. The outcome from instance segmentation could be potentially used as the input of trackers for trac…
Automotive radar has increasingly attracted attention due to growing interest in autonomous driving technologies. Acquiring situational awareness using multimodal data collected at high sampling rates by various sensing devices including cameras, LiDAR, and radar requires considerable power, memory and compute resources which are often limited at an edge device. In this paper, we present a novel adaptive radar sub-sampling algorithm designed to identify regions that require more detailed/accurate reconstruction based on prior environmental conditions' knowledge, enabling near-optimal performan…
Graz University of Technology（格拉茨技术大学）；Infineon Technologies Austria AG（英飞凌科技奥地利股份公司）
Driver assistance systems as well as autonomous cars have to rely on sensors to perceive their environment. A heterogeneous set of sensors is used to perform this task robustly. Among them, radar sensors are indispensable because of their range resolution and the possibility to directly measure velocity. Since more and more radar sensors are deployed on the streets, mutual interference must be dealt with. In the so far unregulated automotive radar frequency band, a sensor must be capable of detecting, or even mitigating the harmful effects of interference, which include a decreased detection s…
University Politehnica of Bucharest（布加勒斯特理工大学）；University of Bucharest（布加勒斯特大学）
Radar sensors are gradually becoming a wide-spread equipment for road vehicles, playing a crucial role in autonomous driving and road safety. The broad adoption of radar sensors increases the chance of interference among sensors from different vehicles, generating corrupted range profiles and range-Doppler maps. In order to extract distance and velocity of multiple targets from range-Doppler maps, the interference affecting each range profile needs to be mitigated. In this paper, we propose a fully convolutional neural network for automotive radar interference mitigation. In order to train our…
Simulation of radar cross-sections (RCS) of pedestrians at automotive radar frequencies forms a key tool for software verification test beds for advanced driver assistance systems. Two commonly used simulation methods are: the computationally simple scattering center model of dynamic humans; and the shooting and bouncing ray technique based on geometric optics. The latter technique is more accurate but due to its computational complexity, it is usually used only for modeling scattered returns of still human poses. In this work, we combine the two methods in a linear regression framework to acc…
University of Ottawa（渥太华大学）；Sensor Cortek Inc.（Sensor Cortek公司）；Valeo（法雷奥）
In this work, we propose the use of radar with advanced deep segmentation models to identify open space in parking scenarios. A publically available dataset of radar observations called SCORP was collected. Deep models are evaluated with various radar input representations. Our proposed approach achieves low memory usage and real-time processing speeds, and is thus very well suited for embedded deployment.
Nowadays, mutual interference among automotive radars has become a problem of wide concern. In this paper, a decentralized spectrum allocation approach is presented to avoid mutual interference among automotive radars. Although decentralized spectrum allocation has been extensively studied in cognitive radio sensor networks, two challenges are observed for automotive sensors using radar. First, the allocation approach should be dynamic as all radars are mounted on moving vehicles. Second, each radar does not communicate with the others so it has quite limited information. A machine learning te…
We present a framework for simulating realistic inverse synthetic aperture radar images of automotive targets at millimeter wave frequencies. The model incorporates radar scattering phenomenology of commonly found vehicles along with range-Doppler based clutter and receiver noise. These images provide insights into the physical dimensions of the target, the number of wheels and the trajectory undertaken by the target. The model is experimentally validated with measurement data gathered from an automotive radar. The images from the simulation database are subsequently classified using both trad…
We present an ultra-short range IEEE 802.11ad-based automotive joint radar-communications (JRC) framework, wherein we improve the radar's Doppler resilience by incorporating Prouhet-Thue-Morse sequences in the preamble. The proposed processing reveals detailed micro-features of common automotive objects verified through extended scattering center models of animated pedestrian, bicycle, and car targets. Numerical experiments demonstrate $2.5$% reduction in the probability-of-false-alarm at low signal-to-noise-ratios and improvement in the peak-to-sidelobe level dynamic range up to Doppler veloc…
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；University of Kassel（卡塞尔大学）
Radar-based road user detection is an important topic in the context of autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to refine during subsequent signal processing. On the other hand, a new sensor generation is waiting in the wings for its application in this challenging field. In this article, two sensors of different radar generations are evaluated against each other. The evaluation criterion is the performance on moving road user object detection and classification tasks. To this end, two data …
University Politehnica of Bucharest（布加勒斯特理工大学）；University of Bucharest（布加勒斯特大学）；Weizmann Institute of Science（魏茨曼科学研究所）
In autonomous driving, radar systems play an important role in detecting targets such as other vehicles on the road. Radars mounted on different cars can interfere with each other, degrading the detection performance. Deep learning methods for automotive radar interference mitigation can succesfully estimate the amplitude of targets, but fail to recover the phase of the respective targets. In this paper, we propose an efficient and effective technique based on unfolded robust Principal Component Analysis (RPCA) that is able to estimate both amplitude and phase in the presence of interference. …
Mutual interference in automotive radar scenarios is going to become a major concern as the density of vehicles with radar sensors in the roads increases. The present work tackles the problem of frequency modulated continuous wave (FMCW) to FMCW and continuous wave interference. In this context, we propose a signal processing technique to blindly identify and remove interference by using the fast Orthogonal Matching Pursuit (OMP) algorithm to project the interference signals in a reduced chirplet basis, and separate it from the target signal with minimal loss of information. Significant reduct…
Automotive radars at the Terahertz (THz) frequency band have the potential to be compact and lightweight while providing high (nearly-optical) angular resolution. In this paper, we propose a bistatic THz automotive radar that employs the recently proposed orthogonal chirp division multiplexing (OCDM) multi-carrier waveform. As a stand-alone communications waveform, OCDM has been investigated for robustness against interference in time-frequency selective channels. The THz-band path loss, and, hence, radar signal bandwidth, are range-dependent. We address this unique feature through a multi-car…
Due to the rapid increase in 76 GHz automotive spectrum use in recent years, wireless interference is becoming a legitimate area of concern. However, the recent rise in interest of automated vehicles (AVs) has also spurred new growth and adoption of low frequency vehicle-to-everything (V2X) communications in and around the 5.8 GHz unlicensed bands, opening the possibility for communications spectrum reuse in the form of joint radar-communications (JRC). In this work, we present a low frequency 5.9 GHz side-looking polarimetric synthetic aperture radar (SAR) for automotive use, utilizing a rang…
Conventional sensing applications rely on electromagnetic far-field channel models with plane wave propagation. However, recent ultra-short-range automotive radar applications at upper millimeter-wave or low terahertz (THz) frequencies envisage operation in the near-field region, where the wavefront is spherical. Unlike far-field, the near-field beampattern is dependent on both range and angle, thus requiring a different approach to waveform design. For the first time in the literature, we adopt the beampattern matching approach to design unimodular waveforms for THz automotive radars with low…
New York University（纽约大学）；NXP Semiconductors（恩智浦半导体）
Among various sensors for assisted and autonomous driving systems, automotive radar has been considered as a robust and low-cost solution even in adverse weather or lighting conditions. With the recent development of radar technologies and open-sourced annotated data sets, semantic segmentation with radar signals has become very promising. However, existing methods are either computationally expensive or discard significant amounts of valuable information from raw 3D radar signals by reducing them to 2D planes via averaging. In this work, we introduce ERASE-Net, an Efficient RAdar SEgmentation…
Automotive targets undergoing turns in road junctions offer large synthetic apertures over short dwell times to automotive radars that can be exploited for obtaining fine cross-range resolution. Likewise, the wide bandwidths of the automotive radar signal yield high-range resolution profiles. Together, they are exploited for generating inverse synthetic aperture radar (ISAR) images that offer rich information regarding the target vehicle's size, shape, and trajectory which is useful for object recognition and classification. However, a key requirement for ISAR is translation motion compensatio…
The interest of the automotive industry has progressively focused on subjects related to driver assistance systems as well as autonomous cars. Cars combine a variety of sensors to perceive their surroundings robustly. Among them, radar sensors are indispensable because of their independence of lighting conditions and the possibility to directly measure velocity. However, radar interference is an issue that becomes prevalent with the increasing amount of radar systems in automotive scenarios. In this paper, we address this issue for frequency modulated continuous wave (FMCW) radars with fully c…
In the automotive sector, both radars and wireless communication are susceptible to interference. However, combining the radar and communication systems, i.e., radio frequency (RF) communications and sensing convergence, has the potential to mitigate interference in both systems. This article analyses the mutual interference of spectrally coexistent frequency modulated continuous wave (FMCW) radar and communication systems in terms of occurrence probability and impact, and introduces RadChat, a distributed networking protocol for mitigation of interference among FMCW based automotive radars, i…
We consider the problem of range-Doppler imaging using one-bit automotive LFMCW1 or PMCW radar that utilizes one-bit ADC sampling with time-varying thresholds at the receiver. The one-bit sampling technique can significantly reduce the cost as well as the power consumption of automotive radar systems. We formulate the one-bit LFMCW/PMCW radar rangeDoppler imaging problem as one-bit sparse parameter estimation. The recently proposed hyperparameter-free (and hence user friendly) weighted SPICE algorithms, including SPICE, LIKES, SLIM and IAA, achieve excellent parameter estimation performance fo…
This paper focuses on the gridless direction-of-arrival (DoA) estimation for data acquired by non-uniform linear arrays (NLAs) in automotive applications. Atomic norm minimization (ANM) is a promising gridless sparse recovery algorithm under the Toeplitz model and solved by convex relaxation, thus it is only applicable to uniform linear arrays (ULAs) with array manifolds having a Vandermonde structure. In automotive applications, it is essential to apply the gridless DoA estimation to NLAs with arbitrary geometry with efficiency. In this paper, a fast ANM-based gridless DoA estimation algorith…
We present an approach to automatically generate semantic labels for real recordings of automotive range-Doppler (RD) radar spectra. Such labels are required when training a neural network for object recognition from radar data. The automatic labeling approach rests on the simultaneous recording of camera and lidar data in addition to the radar spectrum. By warping radar spectra into the camera image, state-of-the-art object recognition algorithms can be applied to label relevant objects, such as cars, in the camera image. The warping operation is designed to be fully differentiable, which all…
In automotive systems, a radar is a key component of autonomous driving. Using transmit and reflected radar signal by a target, we can capture the target range and velocity. However, when interference signals exist, noise floor increases and it severely affects the detectability of target objects. For these reasons, previous studies have been proposed to cancel interference or reconstruct original signals. However, the conventional signal processing methods for canceling the interference or reconstructing the transmit signals are difficult tasks, and also have many restrictions. In this work, …
Automotive radar is a key component in an ADAS. The increasing number of radars implemented in vehicles makes interference between them a noteworthy issue. One method of interference mitigation is to limit the TBP of radar waveforms. However, the problems of how much TBP is necessary and how to optimally utilize the limited TBP have not been addressed. We take CWS as an example and propose a method of designing the radar waveform parameters oriented by the performance of CWS We propose a metric to quantify the CWS performance and study how the radar waveform parameters (bandwidth and duration)…
This paper describes important considerations and challenges associated with online reinforcement-learning based waveform selection for target identification in frequency modulated continuous wave (FMCW) automotive radar systems. We present a novel learning approach based on satisficing Thompson sampling, which quickly identifies a waveform expected to yield satisfactory classification performance. We demonstrate through measurement-level simulations that effective waveform selection strategies can be quickly learned, even in cases where the radar must select from a large catalog of candidate …
Automotive synthetic aperture radar (SAR) can achieve a significant angular resolution enhancement for detecting static objects, which is essential for automated driving. Obtaining high resolution SAR images requires precise ego vehicle velocity estimation. A small velocity estimation error can result in a focused SAR image with objects at offset angles. In this paper, we consider an automotive SAR system that produces SAR images of static objects based on ego vehicle velocity estimation from the radar return signal without the overhead in complexity and cost of using an auxiliary global navig…
Millimeter-wave (mmWave) radars are being increasingly integrated in commercial vehicles to support new Adaptive Driver Assisted Systems (ADAS) features that require accurate location and Doppler velocity estimates of objects, independent of environmental conditions. To explore radar-based ADAS applications, we have updated our test-bed with Texas Instrument's 4-chip cascaded FMCW radar (TIDEP-01012) that forms a non-uniform 2D MIMO virtual array. In this paper, we develop the necessary received signal models for applying different direction of arrival (DoA) estimation algorithms and experimen…
The time division multiple access (TDMA) technique has been applied in automotive multiple-input multiple-output (MIMO) radar. However, it suffers from the transmit energy loss, and as a result the parameter estimation performance degradation when the number of transmit elements increases. To tackle these problem, a transmit beamspace (TB) Doppler division multiple access (DDMA) approach is proposed. First, a phase modulation matrix with empty Doppler spectrum is introduced. By exploiting the empty Doppler spectrum, a test function based on sequential detection is developed to mitigate the Dop…
Displaced automotive sensor imaging exploits joint processing of the data acquired from multiple radar units, each of which may have limited individual resources, to enhance the localization accuracy. Prior works either consider perfect synchronization among the sensors, employ single antenna radars, entail high processing cost, or lack performance analyses. Contrary to these works, we develop a displaced multiple-input multiple-output (MIMO) frequency-modulated continuous-wave (FMCW) radar signal model under coarse synchronization with only frame-level alignment. We derive Bayesian performanc…
We propose a design strategy for optimizing antenna positions in linear arrays for far-field Direction of Arrival (DoA) estimation of narrow-band sources in collocated MIMO radar. Our methodology allows to consider any spatial constraints and number of antennas, using as optimization function the Weiss- Weinstein bound formulated for an observation model with random target phase and known SNR, over a pre-determined Field-of-View (FoV). Optimized arrays are calculated for the typical case of a 77GHz MIMO radar of 3Tx and 4Rx channels. Simulations demonstrate a performance improvement of the pro…
The concept of cognitive radar (CR) enables radar systems to achieve intelligent adaption to a changeable environment with feedback facility from receiver to transmitter. However, the implementation of CR in a fast-changing environment usually requires a well-known environmental model. In our work, we stress the learning ability of CR in an unknown environment using a combination of CR and reinforcement learning (RL), called RL-CR. Less or no model of the environment is required. We also apply the general RL-CR to a specific problem of automotive radar spectrum allocation to mitigate mutual in…
Automotive radar sensors output a lot of unwanted clutter or ghost detections, whose position and velocity do not correspond to any real object in the sensor's field of view. This poses a substantial challenge for environment perception methods like object detection or tracking. Especially problematic are clutter detections that occur in groups or at similar locations in multiple consecutive measurements. In this paper, a new algorithm for identifying such erroneous detections is presented. It is mainly based on the modeling of specific commonly occurring wave propagation paths that lead to cl…
The mutual interference between similar radar systems can result in reduced radar sensitivity and increased false alarm rates. To address the synchronous and asynchronous interference mitigation problems in similar radar systems, we first propose herein two slow-time coding schemes to modulate the pulses within a coherent processing interval (CPI) for a single-input-single-output (SISO) scenario. Specifically, the first coding scheme relies on Doppler shifting and the second one is devised based on an optimization approach. We further extend our discussion to the more general case of multiple-…
In this paper, constant false alarm rate (CFAR) detector-based approaches are proposed for interference mitigation of Frequency modulated continuous wave (FMCW) radars. The proposed methods exploit the fact that after dechirping and low-pass filtering operations the targets' beat signals of FMCW radars are composed of exponential sinusoidal components while interferences exhibit short chirp waves within a sweep. The spectra of interferences in the time-frequency ($t$-$f$) domain are detected by employing a 1-D CFAR detector along each frequency bin and then the detected map is dilated as a mas…
In recent years, Bayes filter methods in the labeled random finite set formulation have become increasingly powerful in the multi-target tracking domain. One of the latest outcomes is the Generalized Labeled Multi-Bernoulli (GLMB) filter which allows for stable cardinality and target state estimation as well as target identification in a unified framework. In contrast to the initial context of the GLMB filter, this paper makes use of it in the Track-Before-Detect (TBD) scheme and thus, avoids information loss due to thresholding and other data preprocessing steps. This paper provides a TBD GLM…
University of Washington（华盛顿大学）；Silkwave Holdings（矽控控股）
Millimeter-wave (mmW) radars are being increasingly integrated in commercial vehicles to support new Adaptive Driver Assisted Systems (ADAS) for its ability to provide high accuracy location, velocity, and angle estimates of objects, largely independent of environmental conditions. Such radar sensors not only perform basic functions such as detection and ranging/angular localization, but also provide critical inputs for environmental perception via object recognition and classification. To explore radar-based ADAS applications, we have assembled a lab-scale frequency modulated continuous wave …
To accurately estimate locations and velocities of surrounding targets (cars) is crucial for advanced driver assistance systems based on radar sensors. In this paper we derive methods for fusing data from multiple radar sensors in order to improve the accuracy and robustness of such estimates. First we pose the target estimation problem as a multivariate multidimensional spectral estimation problem. The problem is multivariate since each radar sensor gives rise to a measurement channel. Then we investigate how the use of the cross-spectra affects target estimates. We see that the use of the ma…
