In conventional machine learning applications, each data attribute is assumed to be orthogonal to others. Namely, every pair of dimension is orthogonal to each other and thus there is no distinction of in-between relations of dimensions. However, this is certainly not the case in real world signals which naturally originate from a spatio-temporal configuration. As a result, the conventional vectorization process disrupts all of the spatio-temporal information about the order/place of data whether it be $1$D, $2$D, $3$D, or $4$D. In this paper, the problem of orthogonality is first investigated through conventional $k$-means of images, where images are to be processed as vectors. As a solution, shift-invariant $k$-means is proposed in a novel framework with the help of sparse representations. A generalization of shift-invariant $k$-means, convolutional dictionary learning, is then utilized as an unsupervised feature extraction method for classification. Experiments suggest that Gabor feature extraction as a simulation of shallow convolutional neural networks provides a little better performance compared to convolutional dictionary learning. Many alternatives of convolutional-logic are also discussed for spatio-temporal information preservation, including a spatio-temporal hypercomplex encoding scheme.
When sensors collect spatio-temporal data in a large geographical area, the existence of missing data cannot be escaped. Missing data negatively impacts the performance of data analysis and machine learning algorithms. In this paper, we study deep autoencoders for missing data imputation in spatio-temporal problems. We propose a convolution bidirectional-LSTM for capturing spatial and temporal patterns. Moreover, we analyze an autoencoder's latent feature representation in spatio-temporal data and illustrate its performance for missing data imputation. Traffic flow data are used for evaluation…
Spatio-temporal data is intrinsically high dimensional, so unsupervised modeling is only feasible if we can exploit structure in the process. When the dynamics are local in both space and time, this structure can be exploited by splitting the global field into many lower-dimensional "light cones". We review light cone decompositions for predictive state reconstruction, introducing three simple light cone algorithms. These methods allow for tractable inference of spatio-temporal data, such as full-frame video. The algorithms make few assumptions on the underlying process yet have good predictiv…
Peking University（北京大学）；Beijing Institute of Big Data Research（北京大数据研究院）
The spatio-temporal graph learning is becoming an increasingly important object of graph study. Many application domains involve highly dynamic graphs where temporal information is crucial, e.g. traffic networks and financial transaction graphs. Despite the constant progress made on learning structured data, there is still a lack of effective means to extract dynamic complex features from spatio-temporal structures. Particularly, conventional models such as convolutional networks or recurrent neural networks are incapable of revealing the temporal patterns in short or long terms and exploring …
We address the problem of predicting spatio-temporal processes with temporal patterns that vary across spatial regions, when data is obtained as a stream. That is, when the training dataset is augmented sequentially. Specifically, we develop a localized spatio-temporal covariance model of the process that can capture spatially varying temporal periodicities in the data. We then apply a covariance-fitting methodology to learn the model parameters which yields a predictor that can be updated sequentially with each new data point. The proposed method is evaluated using both synthetic and real cli…
Penn State University（宾夕法尼亚州立大学）；Hitachi America, Ltd.（日立美国有限公司）
Explosive growth in spatio-temporal data and its wide range of applications have attracted increasing interests of researchers in the statistical and machine learning fields. The spatio-temporal regression problem is of paramount importance from both the methodology development and real-world application perspectives. Given the observed spatially encoded time series covariates and real-valued response data samples, the goal of spatio-temporal regression is to leverage the temporal and spatial dependencies to build a mapping from covariates to response with minimized prediction error. Prior art…
Artificial Intelligence Department Universidad Nacional de Educación a Distancia — UNED（西班牙马德里国立远程教育大学（UNED）人工智能系）
Spatio-temporal forecasting is an open research field whose interest is growing exponentially. In this work we focus on creating a complex deep neural framework for spatio-temporal traffic forecasting with comparatively very good performance and that shows to be adaptable over several spatio-temporal conditions while remaining easy to understand and interpret. Our proposal is based on an interpretable attention-based neural network in which several modules are combined in order to capture key spatio-temporal time series components. Through extensive experimentation, we show how the results of …
As the role played by statistical and computational sciences in climate and environmental modelling and prediction becomes more important, Machine Learning researchers are becoming more aware of the relevance of their work to help tackle the climate crisis. Indeed, being universal nonlinear function approximation tools, Machine Learning algorithms are efficient in analysing and modelling spatially and temporally variable environmental data. While Deep Learning models have proved to be able to capture spatial, temporal, and spatio-temporal dependencies through their automatic feature representa…
Models for fitting spatio-temporal point processes should incorporate spatio-temporal inhomogeneity and allow for different types of interaction between points (clustering or regularity). This paper proposes an extension of the spatial multi-scale area-interaction model to a spatio-temporal framework. This model allows for interaction between points at different spatio-temporal scales and the inclusion of covariates. We fit the proposed model to varicella cases registered during 2013 in Valencia, Spain. The fitted model indicates small scale clustering and regularity for higher spatio-temporal…
Spatio-temporal data and processes are prevalent across a wide variety of scientific disciplines. These processes are often characterized by nonlinear time dynamics that include interactions across multiple scales of spatial and temporal variability. The data sets associated with many of these processes are increasing in size due to advances in automated data measurement, management, and numerical simulator output. Non- linear spatio-temporal models have only recently seen interest in statistics, but there are many classes of such models in the engineering and geophysical sciences. Tradi- tion…
This paper defines the class of càdlàg functional marked point processes (CFMPPs). These are (spatio-temporal) point processes marked by random elements which take values in a càdlàg function space, i.e. the marks are given by càdlàg stochastic processes. We generalise notions of marked (spatio-temporal) point processes and indicate how this class, in a sensible way, connects the point process framework with the random fields framework. We also show how they can be used to construct a class of spatio-temporal Boolean models, how to construct different classes of these models by choosing specif…
Determinantal point processes are models for regular spatial point patterns, with appealing probabilistic properties. We present their spatio-temporal counterparts and give examples of these models, based on spatio-temporal covariance functions which are separable and non-separable in space and time.
Modeling and predicting solar events, particularly the solar ramping event, is critical for improving situational awareness for solar power generation systems. It has been acknowledged that weather conditions such as temperature, humidity, and cloud density can significantly impact the emergence and position of solar ramping events. As a result, modeling these events with complex spatio-temporal correlations is highly challenging. To tackle the question, we adopt a novel spatio-temporal categorical point process model, which intuitively and effectively addresses correlation and interaction amo…
Fine particulate matter (PM$_{2.5}$) has become a great concern worldwide due to its adverse health effects. PM$_{2.5}$ concentrations typically exhibit complex spatio-temporal variations. Both the mean and the spatio-temporal dependence evolve with time due to seasonality, which makes the statistical analysis of PM$_{2.5}$ challenging. In geostatistics, Gaussian process is a powerful tool for characterizing and predicting such spatio-temporal dynamics, for which the specification of a spatio-temporal covariance function is the key. While the extant literature offers a wide range of choices fo…
We propose a new centered autologistic spatio-temporal model for binary data on a lattice. The centering allows the interpretation of the autoregression coefficients in separating the large scale structure of the model corresponding to an expected mean and the small-scale structure corresponding to the auto-correlation. We discuss the existence of the joint law of the process and show by simulation the interest of this kind of centering. We propose and show the efficiency of the maximum pseudo-likelihood estimator and also a method to choose the best structure of neighborhood. Method is applie…
There are many data sources available that report related variables of interest that are also referenced over geographic regions and time; however, there are relatively few general statistical methods that one can readily use that incorporate these multivariate-spatio-temporal dependencies. As such, we introduce the multivariate-spatio-temporal mixed effects model (MSTM) to analyze areal data with multivariate-spatio-temporal dependencies. The proposed MSTM extends the notion of Moran's I basis functions to the multivariate-spatio-temporal setting. This extension leads to several methodologica…
Spatial and spatio-temporal single-structure point process models are widely used in epidemiology, biology, ecology, seismology... . However, most natural phenomena present multiple interaction structure or exhibit dependence at multiple scales in space and/or time, leading to define new spatial and spatio-temporal multi-structure point process models. In this paper, we investigate and review such multi-structure point process models mainly based on Gibbs and Cox processes.
This paper proposes a log-linear model for the latent intensity functions of a replicated spatio-temporal point process. By simultaneously fitting correlated spatial and temporal Karhunen-Loève expansions, the model produces spatial and temporal components that are usually easy to interpret and capture the most important modes of variation and spatio-temporal correlation of the process. The asymptotic distribution of the estimators is derived. The finite sample properties are studied by simulations. As an example of application, we analyze bike usage patterns on the Divvy bike sharing system o…
We consider a stationary spatio-temporal random process and assume that we have a sample. By defining a sequence of discrete Fourier transforms at canonical frequencies at each location, and using these complex valued random varables as observed sample, we obtain expressions for the spatio-temporal covariance functions and the spectral density functions of the spatio-temporal random processes. These spectra correspond to non separable class of random processes. The spatio-temporal covariance functions, obtained here are functions of the spatial distance and the temporal frequency and are simil…
Understanding and predicting environmental phenomena often requires the construction of spatio-temporal statistical models, which are typically Gaussian processes. A common assumption made on Gaussian processes is that of covariance stationarity, which is unrealistic in many geophysical applications. In this article, we introduce a deep-learning-inspired approach to construct descriptive nonstationary spatio-temporal models by modeling stationary processes on warped spatio-temporal domains. The warping functions we use are constructed using several simple injective warping units which, when co…
In the absence of Gaussianity assumptions without disturbing spatial continuity interpolating along the whole spatial surface for different time lags is challenging. The past researchers pay enough attention to Spatio-temporal interpolation ignoring the dynamic behavior of a spatial mean function, threshold distance, and direction of maintaining spatial continuity. Therefore, we employ hierarchical spatial clustering (HSC) to preserve local spatial stationarity. This research work introduces a hybrid extreme valued copula-based Spatio-temporal interpolation algorithm. Spatial dependence is cap…
Spatio-temporal hierarchical modeling is an extremely attractive way to model the spread of crime or terrorism data over a given region, especially when the observations are counts and must be modeled discretely. The spatio-temporal diffusion is placed, as a matter of convenience, in the process model allowing for straightforward estimation of the diffusion parameters through Bayesian techniques. However, this method of modeling does not allow for the existence of self-excitation, or a temporal data model dependency, that has been shown to exist in criminal and terrorism data. In this manuscri…
This paper contributes to the multivariate analysis of marked spatio-temporal point process data by introducing different partial point characteristics and extending the spatial dependence graph model formalism. Our approach yields a unified framework for different types of spatio-temporal data including both, purely qualitatively (multivariate) cases and multivariate cases with additional quantitative marks. The proposed graphical model is defined through partial spectral density characteristics, it is highly computationally efficient and reflects the conditional similarity among sets of spat…
Spatio-temporal covariances are important for describing the spatio-temporal variability of underlying random processes in geostatistical data. For second-order stationary processes, there exist subclasses of covariance functions that assume a simpler spatio-temporal dependence structure with separability and full symmetry. However, it is challenging to visualize and assess separability and full symmetry from spatio-temporal observations. In this work, we propose a functional data analysis approach that constructs test functions using the cross-covariances from time series observed at each pai…
To study the impact of climate variables on morbidity of some diseases in Mexico, we propose a spatio-temporal varying coefficients regression model. For that we introduce a new spatio-temporal dependent process prior, in a Bayesian context, with identically distributed normal marginal distributions and joint multivariate normal distribution. We study its properties and characterise the dependence induced. Our results show that the effect of climate variables, on the incidence of specific diseases, is not constant across space and time and our proposed model is able to capture and quantify tho…
The importance of modelling temperature fields goes beyond the need to understand a region's climate and serves too as a starting point for understanding their socioeconomic, and health consequences. The topography of the study region contributes much to the complexity of modelling these fields and demands flexible spatio-temporal models that are able to handle nonstationarity and changes in trend. In this paper, we develop a flexible stochastic spatio-temporal model for daily temperatures in the Pacific Northwest, and describe a methodology for performing Bayesian spatial prediction. A novel …
Models that capture the spatial and temporal dynamics are applicable in many science fields. Non-separable spatio-temporal models were introduced in the literature to capture these features. However, these models are generally complicated in construction and interpretation. We introduce a class of non-separable Transformed Gaussian Markov Random Fields (TGMRF) in which the dependence structure is flexible and facilitates simple interpretations concerning spatial, temporal and spatio-temporal parameters. Moreover, TGMRF models have the advantage of allowing specialists to define any desired mar…
In spatio-temporal point pattern analysis, one of the main statistical objectives is to estimate the first-order intensity function, i.e., the expected number of points per unit area and unit time. This estimation is usually carried out non-parametrically through kernel functions, where one of the most frequent handicaps is the selection of kernel bandwidths prior to estimation. This work presents an intensity estimation mechanism in which the spatial and temporal bandwidths change at each data point in a spatio-temporal point pattern. This class of estimators is called adaptive estimators, an…
Spatio-temporal models are widely used in many research areas including ecology. The recent proliferation of the use of in-situ sensors in streams and rivers supports space-time water quality modelling and monitoring in near real-time. A new family of spatio-temporal models is introduced. These models incorporate spatial dependence using stream distance while temporal autocorrelation is captured using vector autoregression approaches. Several variations of these novel models are proposed using a Bayesian framework. The results show that our proposed models perform well using spatio-temporal da…
I present an approach for modeling areal spatial covariance by considering the stationary distribution of a spatio-temporal Markov random walk. In the areal data case, this stationary distribution corresponds to an intrinsic simultaneous autoregressive (SAR) model for spatial correlation, and provides a principled approach to specifying areal spatial models when a spatio-temporal generating process can be assumed. I apply the approach to a study of spatial genetic variation of trout in a stream network in Connecticut, USA, and a study of crime rates in neighborhoods of Columbus, OH, USA.
Many physical processes involve spatio-temporal observations, which can be studied at different spatial and temporal scales. For example, rainfall data measured daily by rain gauges can be considered at daily, monthly or annual temporal scales, and local, grid-wise, region-wise or country-wise spatial scales. In this work, we focus on detection of anomalies in such multi-scale spatio-temporal data. We consider an anomaly as an event where the measured values over a spatio-temporally extended region are significantly different from their long-term means. However we aim to avoid setting any thre…
We restrict our attention to space-time point pattern data for which we have a single realisation within a finite region. Second-order characteristics are used to analyse the spatio-temporal structure of the underlying point process. In particular, the space-time inhomogeneous pair correlation function and $K$-function measure the spatio-temporal clustering/regularity and the spatio-temporal interaction, and thus help with model choice. Non-parametric estimators of the second-order characteristics require information from outside the study region, resulting to the so-called edge effects which …
It is of utmost importance to have a clear understanding of the status of air pollution and to provide forecasts and insights about the air quality to the general public and researchers in environmental studies. Previous studies of spatio-temporal models showed that even a short-term exposure to high concentrations of atmospheric fine particulate matters can be hazardous to the health of ordinary people. In this study, we develop a spatio-temporal model with space-time interaction for air pollution data. The proposed model uses a parametric space-time interaction component along with the spati…
We propose an approximation to the forward-filter-backward-sampler (FFBS) algorithm for large-scale spatio-temporal smoothing. FFBS is commonly used in Bayesian statistics when working with linear Gaussian state-space models, but it requires inverting covariance matrices which have the size of the latent state vector. The computational burden associated with this operation effectively prohibits its applications in high-dimensional settings. We propose a scalable spatio-temporal FFBS approach based on the hierarchical Vecchia approximation of Gaussian processes, which has been previously succes…
Predicting ambulance demand accurately at fine time and location scales is critical for ambulance fleet management and dynamic deployment. Large-scale datasets in this setting typically exhibit complex spatio-temporal dynamics and sparsity at high resolutions. We propose a predictive method using spatio-temporal kernel density estimation (stKDE) to address these challenges, and provide spatial density predictions for ambulance demand in Toronto, Canada as it varies over hourly intervals. Specifically, we weight the spatial kernel of each historical observation by its informativeness to the cur…
I begin my discussion by summarizing the methodology proposed and new distributional results on multivariate log-Gamma derived in the paper. Then, I draw an interesting connection between their work with mean field variational Bayes. Lastly, I make some comments on the simulation results and the performance of the proposed Poisson multivariate spatio-temporal mixed effects model (P-MSTM).
We introduce WECS (Wavelet Energies Correlation Sreening), an unsupervised sparse procedure to detect spatio-temporal change points on multi-temporal SAR (POLSAR) images or even on sequences of very high resolution images. The procedure is based on wavelet approximation for the multi-temporal images, wavelet energy apportionment, and ultra-high dimensional correlation screening for the wavelet coefficients. We present two complimentary wavelet measures in order to detect sudden and/or cumulative changes, as well as for the case of stationary or non-stationary multi-temporal images. We show WEC…
A class of physics-informed spatio-temporal models has recently been proposed for modeling spatio-temporal processes governed by advection-diffusion equations. The central idea is to approximate the process by a truncated Fourier series and let the governing physics determine the dynamics of the spectral coefficients. However, because many spatio-temporal processes in real applications are non-periodic with boundary discontinuities, the well-known Gibbs phenomenon and ripple artifact almost always exist in the outputs generated by such models due to truncation of the Fourier series. Hence, the…
The stopp R package deals with spatio-temporal point processes which might have occurred on the Euclidean space or on some specific linear networks such as roads of a city. The package contains functions to summarize, plot, and perform different kinds of analyses on point processes, mainly following the methods proposed in some recent papers in the stream of scientific literature. The main topics of such works, and of the package in turn, include modeling, statistical inference, and simulation issues on spatio-temporal point processes on Euclidean space and linear networks, with a focus on the…
We consider modeling and prediction of Capelin distribution in the Barents sea based on zero-inflated count observation data that vary continuously over a specified survey region. The model is a mixture of two components; a one-point distribution at the origin and a Poisson distribution with spatio-temporal intensity, where both intensity and mixing proportions are modeled by some auxiliary variables and unobserved spatio-temporal effects. The spatio-temporal effects are modeled by a dynamic linear model combined with the predictive Gaussian process. We develop an efficient posterior computati…
We propose a new way of incorporating temporal information present in videos into Spatial Convolutional Neural Networks (ConvNets) trained on images, that avoids training Spatio-Temporal ConvNets from scratch. We describe several initializations of weights in 3D Convolutional Layers of Spatio-Temporal ConvNet using 2D Convolutional Weights learned from ImageNet. We show that it is important to initialize 3D Convolutional Weights judiciously in order to learn temporal representations of videos. We evaluate our methods on the UCF-101 dataset and demonstrate improvement over Spatial ConvNets.
This paper investigates a fully unsupervised statistical method for edge preserving image restoration and compression using a spatial decomposition scheme. Smoothed maximum likelihood is used for local estimation of edge pixels from mixture parametric models of local templates. For the complementary smooth part the traditional L2-variational problem is solved in the Fourier domain with Thin Plate Spline (TPS) regularization. It is well known that naive Fourier compression of the whole image fails to restore a piece-wise smooth noisy image satisfactorily due to Gibbs phenomenon. Images are inte…
Spatio-temporal action detection is an important and challenging problem in video understanding. However, the application of the existing large-scale spatio-temporal action datasets in specific fields is limited, and there is currently no public tool for making spatio-temporal action datasets, it takes a lot of time and effort for researchers to customize the spatio-temporal action datasets, so we propose a multi-Person video dataset Annotation Method of spatio-temporally actions.First, we use ffmpeg to crop the videos and frame the videos; then use yolov5 to detect human in the video frame, a…
National Laboratory for Parallel and Distributed Processing（并行与分布式处理国家重点实验室）；National University of Defense Technology（国防科技大学）
Temporal action localization is an important task of computer vision. Though a variety of methods have been proposed, it still remains an open question how to predict the temporal boundaries of action segments precisely. Most works use segment-level classifiers to select video segments pre-determined by action proposal or dense sliding windows. However, in order to achieve more precise action boundaries, a temporal localization system should make dense predictions at a fine granularity. A newly proposed work exploits Convolutional-Deconvolutional-Convolutional (CDC) filters to upsample the pre…
University of Cincinnati（辛辛那提大学）；University of Minnesota（明尼苏达大学）
Large volumes of spatio-temporal data are increasingly collected and studied in diverse domains including, climate science, social sciences, neuroscience, epidemiology, transportation, mobile health, and Earth sciences. Spatio-temporal data differs from relational data for which computational approaches are developed in the data mining community for multiple decades, in that both spatial and temporal attributes are available in addition to the actual measurements/attributes. The presence of these attributes introduces additional challenges that needs to be dealt with. Approaches for mining spa…
