Synthetic aperture sonar (SAS) requires precise time-of-flight measurements of the transmitted/received waveform to produce well-focused imagery. It is not uncommon for errors in these measurements to be present resulting in image defocusing. To overcome this, an \emph{autofocus} algorithm is employed as a post-processing step after image reconstruction to improve image focus. A particular class of these algorithms can be framed as a sharpness/contrast metric-based optimization. To improve convergence, a hand-crafted weighting function to remove "bad" areas of the image is sometimes applied to the image-under-test before the optimization procedure. Additionally, dozens of iterations are necessary for convergence which is a large compute burden for low size, weight, and power (SWaP) systems. We propose a deep learning technique to overcome these limitations and implicitly learn the weighting function in a data-driven manner. Our proposed method, which we call Deep Autofocus, uses features from the single-look-complex (SLC) to estimate the phase correction which is applied in $k$-space. Furthermore, we train our algorithm on batches of training imagery so that during deployment, only a single iteration of our method is sufficient to autofocus. We show results demonstrating the robustness of our technique by comparing our results to four commonly used image sharpness metrics. Our results demonstrate Deep Autofocus can produce imagery perceptually better than common iterative techniques but at a lower computational cost. We conclude that Deep Autofocus can provide a more favorable cost-quality trade-off than alternatives with significant potential of future research.
Synthetic aperture sonar (SAS) requires precise positional and environmental information to produce well-focused output during the image reconstruction step. However, errors in these measurements are commonly present resulting in defocused imagery. To overcome these issues, an \emph{autofocus} algorithm is employed as a post-processing step after image reconstruction for the purpose of improving image quality using the image content itself. These algorithms are usually iterative and metric-based in that they seek to optimize an image sharpness metric. In this letter, we demonstrate the potenti…
Deep learning has not been routinely employed for semantic segmentation of seabed environment for synthetic aperture sonar (SAS) imagery due to the implicit need of abundant training data such methods necessitate. Abundant training data, specifically pixel-level labels for all images, is usually not available for SAS imagery due to the complex logistics (e.g., diver survey, chase boat, precision position information) needed for obtaining accurate ground-truth. Many hand-crafted feature based algorithms have been proposed to segment SAS in an unsupervised fashion. However, there is still room f…
University of Florida（佛罗里达大学）；University of Missouri（密苏里大学）
Side-look synthetic aperture sonar (SAS) can produce very high quality images of the sea-floor. When viewing this imagery, a human observer can often easily identify various sea-floor textures such as sand ripple, hard-packed sand, sea grass and rock. In this paper, we present the Possibilistic Fuzzy Local Information C-Means (PFLICM) approach to segment SAS imagery into sea-floor regions that exhibit these various natural textures. The proposed PFLICM method incorporates fuzzy and possibilistic clustering methods and leverages (local) spatial information to perform soft segmentation. Results …
Applied Research Laboratory（应用研究实验室）；School of EECS at the Pennsylvania State University（宾夕法尼亚州立大学电子工程与计算机科学学院）
Deep learning has been recently shown to improve performance in the domain of synthetic aperture sonar (SAS) image classification. Given the constant resolution with range of a SAS, it is no surprise that deep learning techniques perform so well. Despite deep learning's recent success, there are still compelling open challenges in reducing the high false alarm rate and enabling success when training imagery is limited, which is a practical challenge that distinguishes the SAS classification problem from standard image classification set-ups where training imagery may be abundant. We address th…
Pennsylvania State University（宾夕法尼亚州立大学）；Applied Research Laboratory（应用研究实验室）
Synthetic aperture sonar (SAS) systems produce high-resolution images of the seabed environment. Moreover, deep learning has demonstrated superior ability in finding robust features for automating imagery analysis. However, the success of deep learning is conditioned on having lots of labeled training data, but obtaining generous pixel-level annotations of SAS imagery is often practically infeasible. This challenge has thus far limited the adoption of deep learning methods for SAS segmentation. Algorithms exist to segment SAS imagery in an unsupervised manner, but they lack the benefit of stat…
Pennsylvania State University（宾夕法尼亚州立大学）；Centre for Maritime Research & Experimentation（海事研究与实验中心）
Object classification in synthetic aperture sonar (SAS) imagery is usually a data starved and class imbalanced problem. There are few objects of interest present among much benign seafloor. Despite these problems, current classification techniques discard a large portion of the collected SAS information. In particular, a beamformed SAS image, which we call a single-look complex (SLC) image, contains complex pixels composed of real and imaginary parts. For human consumption, the SLC is converted to a magnitude-phase representation and the phase information is discarded. Even more problematic, t…
Histogram Layers for Synthetic Aperture Sonar Imagery
Naval Surface Warfare Center Panama City Division（巴拿马城海军水面作战中心分部）；University of Missouri（密苏里大学）；University of Florida（佛罗里达大学）；Texas A&M University（德州农工大学）
Pennsylvania State University（宾夕法尼亚州立大学）；U.S. Naval Research Laboratory（美国海军研究实验室）；Naval Surface Warfare Center Panama City（巴拿马城海军水面作战中心）
Advancements in Sonar image capture have enabled researchers to apply sophisticated object identification algorithms in order to locate targets of interest in images such as mines. Despite progress in this field, modern sonar automatic target recognition (ATR) approaches lack robustness to the amount of noise one would expect in real-world scenarios, the capability to handle blurring incurred from the physics of image capture, and the ability to excel with relatively few training samples. We address these challenges by adapting modern sparsity-based techniques with dictionaries comprising of t…
Using Frame Theoretic Convolutional Gridding for Robust Synthetic Aperture Sonar Imaging
Department of Electrical Engineering, Pennsylvania State University, State College, PA（电气工程系，宾夕法尼亚州立大学，州立学院，PA）；Department of Mathematics, Dartmouth College, Hanover, NH（数学系，达特茅斯学院，汉诺威，NH）；U.S. Naval Research Laboratory, Washington, DC（美国海军研究实验室，华盛顿，DC）
Observations of seabed texture with synthetic aperture sonar are dependent upon several factors. In this work, we focus on collection geometry with respect to isotropic and anisotropic textures. The low grazing angle of the collection geometry, combined with orientation of the sonar path relative to anisotropic texture, poses a significant challenge for image-alignment and other multi-view scene understanding frameworks. We previously proposed using features captured from estimated seabed relief to improve scene understanding. While several methods have been developed to estimate seabed relief…
Applied Research Laboratory, Pennsylvania State University（宾夕法尼亚州立大学应用研究实验室）
Advances in unmanned synthetic aperture sonar (SAS) imaging platforms allow for the simultaneous collection of multiband SAS imagery. The imagery is collected over several octaves and the phenomenology's interactions with the sea floor vary greatly over this range -- higher frequencies resolve proud and fine structure of the seafloor while lower frequencies resolve subsurface features and often induce internal resonance in man-made objects. Currently, analysts examine multiband imagery by viewing a single band at a time. This method makes it difficult to ascertain correlations between any pair…
Arizona State University（亚利桑那州立大学）；Pennsylvania State University（宾夕法尼亚州立大学）
Synthetic aperture sonar (SAS) image resolution is constrained by waveform bandwidth and array geometry. Specifically, the waveform bandwidth determines a point spread function (PSF) that blurs the locations of point scatterers in the scene. In theory, deconvolving the reconstructed SAS image with the scene PSF restores the original distribution of scatterers and yields sharper reconstructions. However, deconvolution is an ill-posed operation that is highly sensitive to noise. In this work, we leverage implicit neural representations (INRs), shown to be strong priors for the natural image spac…
Arizona State University（亚利桑那州立大学）；Pennsylvania State University（宾夕法尼亚州立大学）；NATO STO Centre for Maritime Research & Experimentation（北约科学与技术组织海事研究与实验中心）
Acquisition of Synthetic Aperture Sonar (SAS) datasets is bottlenecked by the costly deployment of SAS imaging systems, and even when data acquisition is possible,the data is often skewed towards containing barren seafloor rather than objects of interest. We present a novel pipeline, called SAS GAN, which couples an optical renderer with a generative adversarial network (GAN) to synthesize realistic SAS images of targets on the seafloor. This coupling enables high levels of SAS image realism while enabling control over image geometry and parameters. We demonstrate qualitative results by presen…
Applied Research Labs Penn State（宾夕法尼亚州立大学应用研究实验室）
Performance of automatic target recognition from synthetic aperture sonar data is heavily dependent on the complexity of the beamformed imagery. Several mechanisms can contribute to this, including unwanted vehicle dynamics, the bathymetry of the scene, and the presence of natural and manmade clutter. To understand the impact of the environmental complexity on image perception, researchers have taken approaches rooted in information theory, or heuristics. Despite these efforts, a quantitative measure for complexity has not been related to the phenomenology from which it is derived. By using su…
University of Florida（佛罗里达大学）；University of Missouri（密苏里大学）
The Possibilistic Fuzzy Local Information C-Means (PFLICM) method is presented as a technique to segment side-look synthetic aperture sonar (SAS) imagery into distinct regions of the sea-floor. In this work, we investigate and present the results of an automated feature selection approach for SAS image segmentation. The chosen features and resulting segmentation from the image will be assessed based on a select quantitative clustering validity criterion and the subset of the features that reach a desired threshold will be used for the segmentation process.
