The Internet-of-Things, complex sensor networks, multi-agent cyber-physical systems are all examples of spatially distributed systems that continuously evolve in time. Such systems generate huge amounts of spatio-temporal data, and system designers are often interested in analyzing and discovering structure within the data. There has been considerable interest in learning causal and logical properties of temporal data using logics such as Signal Temporal Logic (STL); however, there is limited work on discovering such relations on spatio-temporal data. We propose the first set of algorithms for unsupervised learning for spatio-temporal data. Our method does automatic feature extraction from the spatio-temporal data by projecting it onto the parameter space of a parametric spatio-temporal reach and escape logic (PSTREL). We propose an agglomerative hierarchical clustering technique that guarantees that each cluster satisfies a distinct STREL formula. We show that our method generates STREL formulas of bounded description complexity using a novel decision-tree approach which generalizes previous unsupervised learning techniques for Signal Temporal Logic. We demonstrate the effectiveness of our approach on case studies from diverse domains such as urban transportation, epidemiology, green infrastructure, and air quality monitoring.
University of Minnesota, Twin Cities（明尼苏达大学双城分校）；University of Florida（佛罗里达大学）
Spatiotemporal data mining aims to discover interesting, useful but non-trivial patterns in big spatial and spatiotemporal data. They are used in various application domains such as public safety, ecology, epidemiology, earth science, etc. This problem is challenging because of the high societal cost of spurious patterns and exorbitant computational cost. Recent surveys of spatiotemporal data mining need update due to rapid growth. In addition, they did not adequately survey parallel techniques for spatiotemporal data mining. This paper provides a more up-to-date survey of spatiotemporal data …
University of Cincinnati（辛辛那提大学）；University of Minnesota（明尼苏达大学）
Large volumes of spatio-temporal data are increasingly collected and studied in diverse domains including, climate science, social sciences, neuroscience, epidemiology, transportation, mobile health, and Earth sciences. Spatio-temporal data differs from relational data for which computational approaches are developed in the data mining community for multiple decades, in that both spatial and temporal attributes are available in addition to the actual measurements/attributes. The presence of these attributes introduces additional challenges that needs to be dealt with. Approaches for mining spa…
Spatiotemporal data mining (STDM) discovers useful patterns from the dynamic interplay between space and time. Several available surveys capture STDM advances and report a wealth of important progress in this field. However, STDM challenges and problems are not thoroughly discussed and presented in articles of their own. We attempt to fill this gap by providing a comprehensive literature survey on state-of-the-art advances in STDM. We describe the challenging issues and their causes and open gaps of multiple STDM directions and aspects. Specifically, we investigate the challenging issues in re…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Tsinghua University（清华大学）
With the fast development of various positioning techniques such as Global Position System (GPS), mobile devices and remote sensing, spatio-temporal data has become increasingly available nowadays. Mining valuable knowledge from spatio-temporal data is critically important to many real world applications including human mobility understanding, smart transportation, urban planning, public safety, health care and environmental management. As the number, volume and resolution of spatio-temporal datasets increase rapidly, traditional data mining methods, especially statistics based methods for dea…
Development of interpretable machine learning models for clinical healthcare applications has the potential of changing the way we understand, treat, and ultimately cure, diseases and disorders in many areas of medicine. These models can serve not only as sources of predictions and estimates, but also as discovery tools for clinicians and researchers to reveal new knowledge from the data. High dimensionality of patient information (e.g., phenotype, genotype, and medical history), lack of objective measurements, and the heterogeneity in patient populations often create significant challenges in…
Interpretable Data Mining of Follicular Thyroid Cancer Ultrasound Features Using Enhanced Association Rules
Qiuzhen Colledge, Tsinghua University（清华大学邱振学院）；Department of Mathematics, Tsinghua University（清华大学数学系）；Department of General Surgery, Peking University Third Hospital（北京大学第三医院外科部）；Beijing Institute of Mathematical Sciences and Applications(BIMSA)（北京应用数学研究所）
Huawei Technologies Canada（华为技术加拿大公司）；McMaster University（麦克马斯特大学）；University of British Columbia（不列颠哥伦比亚大学）；Simon Fraser University（西蒙弗雷泽大学）；Tianjin University（天津大学）
The popularity of machine learning has increased the risk of unfair models getting deployed in high-stake applications, such as justice system, drug/vaccination design, and medical diagnosis. Although there are effective methods to train fair models from scratch, how to automatically reveal and explain the unfairness of a trained model remains a challenging task. Revealing unfairness of machine learning models in interpretable fashion is a critical step towards fair and trustworthy AI. In this paper, we systematically tackle the novel task of revealing unfair models by mining interpretable evi…
Stanford University（斯坦福大学）；Universidad Catolica de Chile（智利天主教大学）
A key aspect of VQA models that are interpretable is their ability to ground their answers to relevant regions in the image. Current approaches with this capability rely on supervised learning and human annotated groundings to train attention mechanisms inside the VQA architecture. Unfortunately, obtaining human annotations specific for visual grounding is difficult and expensive. In this work, we demonstrate that we can effectively train a VQA architecture with grounding supervision that can be automatically obtained from available region descriptions and object annotations. We also show that…
Machine learning algorithms such as linear regression, SVM and neural network have played an increasingly important role in the process of scientific discovery. However, none of them is both interpretable and accurate on nonlinear datasets. Here we present contextual regression, a method that joins these two desirable properties together using a hybrid architecture of neural network embedding and dot product layer. We demonstrate its high prediction accuracy and sensitivity through the task of predictive feature selection on a simulated dataset and the application of predicting open chromatin …
Geoscience and seismology have utilized the most advanced technologies and equipment to monitor seismic events globally from the past few decades. With the enormous amount of data, modern GPU-powered deep learning presents a promising approach to analyze data and discover patterns. In recent years, there are plenty of successful deep learning models for picking seismic waves. However, forecasting extreme earthquakes, which can cause disasters, is still an underdeveloped topic in history. Relevant research in spatiotemporal dynamics mining and forecasting has revealed some successful prediction…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
Accuracy and interpretability are two essential properties for a crime prediction model. Because of the adverse effects that the crimes can have on human life, economy and safety, we need a model that can predict future occurrence of crime as accurately as possible so that early steps can be taken to avoid the crime. On the other hand, an interpretable model reveals the reason behind a model's prediction, ensures its transparency and allows us to plan the crime prevention steps accordingly. The key challenge in developing the model is to capture the non-linear spatial dependency and temporal p…
L3S Research Center（L3S研究中心）；Leibniz University Hannover（莱布尼茨汉诺威大学）；University of Bonn（波恩大学）
The discovery of spatio-temporal dependencies within urban road networks that cause Recurrent Congestion (RC) patterns is crucial for numerous real-world applications, including urban planning and scheduling of public transportation services. While most existing studies investigate temporal patterns of RC phenomena, the influence of the road network topology on RC is often overlooked. This article proposes the ST-Discovery algorithm, a novel unsupervised spatio-temporal data mining algorithm that facilitates the effective data-driven discovery of RC dependencies induced by the road network top…
Advancing Enterprise Spatio-Temporal Forecasting Applications: Data Mining Meets Instruction Tuning of Language Models For Multi-modal Time Series Analysis in Low-Resource Settings
This thesis focuses on data that has complex spatio-temporal structure and on probabilistic graphical models that learn the structure in an interpretable and scalable manner. We target two research areas of interest: Gaussian graphical models for tensor-variate data and summarization of complex time-varying texts using topic models. This work advances the state-of-the-art in several directions. First, it introduces a new class of tensor-variate Gaussian graphical models via the Sylvester tensor equation. Second, it develops an optimization technique based on a fast-converging proximal alternat…
Experimental data is often affected by uncontrolled variables that make analysis and interpretation difficult. For spatiotemporal systems, this problem is further exacerbated by their intricate dynamics. Modern machine learning methods are particularly well-suited for analyzing and modeling complex datasets, but to be effective in science, the result needs to be interpretable. We demonstrate an unsupervised learning technique for extracting interpretable physical parameters from noisy spatiotemporal data and for building a transferable model of the system. In particular, we implement a physics…
To accommodate the unprecedented increase of commercial airlines over the next ten years, the Next Generation Air Transportation System (NextGen) has been implemented in the USA that records large-scale Air Traffic Management (ATM) data to make air travel safer, more efficient, and more economical. A key role of collaborative decision making for air traffic scheduling and airspace resource management is the accurate prediction of flight delay. There has been a lot of attempts to apply data-driven methods such as machine learning to forecast flight delay situation using air traffic data of depa…
Shanghai Jiao Tong University（上海交通大学）；University of California, Los Angeles（加州大学洛杉矶分校）
In this paper, we present a method to mine object-part patterns from conv-layers of a pre-trained convolutional neural network (CNN). The mined object-part patterns are organized by an And-Or graph (AOG). This interpretable AOG representation consists of a four-layer semantic hierarchy, i.e., semantic parts, part templates, latent patterns, and neural units. The AOG associates each object part with certain neural units in feature maps of conv-layers. The AOG is constructed in a weakly-supervised manner, i.e., very few annotations (e.g., 3-20) of object parts are used to guide the learning of A…
Wayne State University（韦恩州立大学）；DiDi Research America（滴滴美国研究院）；DiDi Research（滴滴研究院）
In this paper, we develop a reinforcement learning (RL) based system to learn an effective policy for carpooling that maximizes transportation efficiency so that fewer cars are required to fulfill the given amount of trip demand. For this purpose, first, we develop a deep neural network model, called ST-NN (Spatio-Temporal Neural Network), to predict taxi trip time from the raw GPS trip data. Secondly, we develop a carpooling simulation environment for RL training, with the output of ST-NN and using the NYC taxi trip dataset. In order to maximize transportation efficiency and minimize traffic …
Dalhousie University（达尔豪斯大学）；The University of Waikato（怀卡托大学）
Due to the surge of spatio-temporal data volume, the popularity of location-based services and applications, and the importance of extracted knowledge from spatio-temporal data to solve a wide range of real-world problems, a plethora of research and development work has been done in the area of spatial and spatio-temporal data analytics in the past decade. The main goal of existing works was to develop algorithms and technologies to capture, store, manage, analyze, and visualize spatial or spatio-temporal data. The researchers have contributed either by adding spatio-temporal support with exis…
Foundation Models for Spatio-Temporal Data Science: A Tutorial and Survey
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Carnegie Mellon University（卡内基梅隆大学）；National University of Singapore（新加坡国立大学）；Griffith University（格里菲斯大学）；East China Normal University（华东师范大学）；University of New South Wales（新南威尔士大学）；Squirrel Ai Learning, USA（美国松鼠人工智能学习公司）；Nanyang Technology University（南洋理工大学）
Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation
University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德河谷分校）；University of Maryland（马里兰大学）；University of Pittsburgh（匹兹堡大学）；Texas A&M University - Corpus Christi（德克萨斯农工大学科珀斯克里斯蒂分校）；Emory University（埃默里大学）；Lehigh University（利哈伊大学）；University of Illinois at Chicago（芝加哥伊利诺伊大学）；University of Southern California（南加州大学）
Xidian University（西安电子科技大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Embedded topic models are able to learn interpretable topics even with large and heavy-tailed vocabularies. However, they generally hold the Euclidean embedding space assumption, leading to a basic limitation in capturing hierarchical relations. To this end, we present a novel framework that introduces hyperbolic embeddings to represent words and topics. With the tree-likeness property of hyperbolic space, the underlying semantic hierarchy among words and topics can be better exploited to mine more interpretable topics. Furthermore, due to the superiority of hyperbolic geometry in representing…
Disentangled representation learning offers useful properties such as dimension reduction and interpretability, which are essential to modern deep learning approaches. Although deep learning techniques have been widely applied to spatio-temporal data mining, there has been little attention to further disentangle the latent features and understanding their contribution to the model performance, particularly their mutual information and correlation across features. In this study, we adopt two state-of-the-art disentangled representation learning methods and apply them to three large-scale public…
Since most machine learning models provide no explanations for the predictions, their predictions are obscure for the human. The ability to explain a model's prediction has become a necessity in many applications including Twitter mining. In this work, we propose a method called Explainable Twitter Mining (Ex-Twit) combining Topic Modeling and Local Interpretable Model-agnostic Explanation (LIME) to predict the topic and explain the model predictions. We demonstrate the effectiveness of Ex-Twit on Twitter health-related data.
University of British Columbia（不列颠哥伦比亚大学）；Tsinghua University（清华大学）
Inspired by the observation that humans are able to process videos efficiently by only paying attention where and when it is needed, we propose an interpretable and easy plug-in spatial-temporal attention mechanism for video action recognition. For spatial attention, we learn a saliency mask to allow the model to focus on the most salient parts of the feature maps. For temporal attention, we employ a convolutional LSTM based attention mechanism to identify the most relevant frames from an input video. Further, we propose a set of regularizers to ensure that our attention mechanism attends to c…
ICST-DNET: An Interpretable Causal Spatio-Temporal Diffusion Network for Traffic Speed Prediction
College of Computer and Software, Hohai University（河海大学计算机与软件学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；College of Internet of Things, Nanjing University of Posts and Telecommunications（南京邮电大学物联网学院）
Harvard University, Institute for Quantitative Social Science（哈佛大学量化社会科学研究所）；Stockholm University Demography Unit（斯德哥尔摩大学人口学单位）；Malmö University, Department of Global Political Studies（马尔默大学全球政治研究系）；University of Catania, Department of Political and Social Sciences（卡塔尼亚大学政治与社会科学系）；University of Toronto, Graduate School of Public and International Affairs（多伦多大学公共与国际事务研究生院）
Department of Electrical & Computer Engineering（电气与计算机工程系）；Department of Applied Mathematics（应用数学系）；University of Washington（华盛顿大学）；Seattle, WA 98195（西雅图, WA 98195（注意：地址信息按要求不应提取，此处仅为示例说明））
Encoder-decoder recurrent neural network models (RNN Seq2Seq) have achieved great success in ubiquitous areas of computation and applications. It was shown to be successful in modeling data with both temporal and spatial dependencies for translation or prediction tasks. In this study, we propose an embedding approach to visualize and interpret the representation of data by these models. Furthermore, we show that the embedding is an effective method for unsupervised learning and can be utilized to estimate the optimality of model training. In particular, we demonstrate that embedding space proj…
Truck platooning refers to a series of trucks driving in close proximity via communication technologies, and it is considered one of the most implementable systems of connected and automated vehicles, bringing huge energy savings and safety improvements. Properly planning platoons and evaluating the potential of truck platooning are crucial to trucking companies and transportation authorities. This study proposes a series of data mining approaches to learn spontaneous truck platooning patterns from massive trajectories. An enhanced map matching algorithm is developed to identify truck headings…
SparseST: Exploiting Data Sparsity in Spatiotemporal Modeling and Prediction
Department of Industrial and Systems Engineering, Rensselaer Polytechnic Institute（工业与系统工程系，罗切斯特理工学院）；Department of Electrical, Computer, and Systems Engineering and the Department of Computer Science, Rensselaer Polytechnic Institute（电气、计算机与系统工程系和计算机科学系，罗切斯特理工学院）；IBM Research（IBM研究院）
Unifying Physics- and Data-Driven Modeling via Novel Causal Spatiotemporal Graph Neural Network for Interpretable Epidemic Forecasting
Frankfurt Institute for Advanced Studies（法兰克福高等研究院）；Goethe Universität Frankfurt（法兰克福大学）；Xidian-FIAS International Joint Research Center（西电-法兰克福高等研究院国际联合研究中心）；School of Science and Engineering, The Chinese University of Hong Kong（香港中文大学（深圳）理工学院）；GSI Helmholtzzentrum für Schwerionenforschung GmbH（GSI亥姆霍兹重离子研究中心）
INESC TEC；University of Porto（波尔图大学）；Universidade da Coruña（拉科鲁尼亚大学）
Many aspects of life are associated with places of human mobility patterns and nowadays we are facing an increase in the pervasiveness of mobile devices these individuals carry. Positioning technologies that serve these devices such as the cellular antenna (GSM networks), global navigation satellite systems (GPS), and more recently the WiFi positioning system (WPS) provide large amounts of spatio-temporal data in a continuous way. Therefore, detecting significant places and the frequency of movements between them is fundamental to understand human behavior. In this paper, we propose a method f…
BEAT-Net: Injecting Biomimetic Spatio-Temporal Priors for Interpretable ECG Diagnosis
School of Information Technology（信息科技学院）；Monash University Malaysia（墨尔本大学马来西亚分校）；Faculty of Biomedical Engineering（生物医学工程学院）；Shenzhen University of Advanced Technology（深圳先进技术大学）
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook
Griffith University（格里菲斯大学）；University of Oxford（牛津大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Zhejiang Normal University（浙江师范大学）；Ant Group（蚂蚁集团）；Alibaba Group（阿里巴巴集团）；Deloitte Service LLP（德勤服务有限责任公司）；The University of Hong Kong（香港大学）；NEC Laboratories America（NEC美国实验室）；A*STAR；National Yang Ming Chiao Tung University（阳明交通大学）；JD Technology（京东科技）；Squirrel Ai Learning
Artificial Intelligence Department Universidad Nacional de Educación a Distancia — UNED（西班牙马德里国立远程教育大学（UNED）人工智能系）
Spatio-temporal forecasting is an open research field whose interest is growing exponentially. In this work we focus on creating a complex deep neural framework for spatio-temporal traffic forecasting with comparatively very good performance and that shows to be adaptable over several spatio-temporal conditions while remaining easy to understand and interpret. Our proposal is based on an interpretable attention-based neural network in which several modules are combined in order to capture key spatio-temporal time series components. Through extensive experimentation, we show how the results of …
Regression modelling of spatiotemporal extreme U.S. wildfires via partially-interpretable neural networks
School of Mathematics, University of Edinburgh（爱丁堡大学数学学院）；Statistics Program, Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division, King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学计算机、电气与数学科学与工程（CEMSE）分部统计项目）
Interpretable Cascading Mixture-of-Experts for Urban Traffic Congestion Prediction
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Didichuxing Co. Ltd（滴滴出行有限公司）
Jožef Stefan International Postgraduate School（约热夫·斯特凡国际研究生院）；Jožef Stefan Institute（约热夫·斯特凡研究所）；University of Nova Gorica（新戈里察大学）
Relational data mining is becoming ubiquitous in many fields of study. It offers insights into behaviour of complex, real-world systems which cannot be modeled directly using propositional learning. We propose Symbolic Graph Embedding (SGE), an algorithm aimed to learn symbolic node representations. Built on the ideas from the field of inductive logic programming, SGE first samples a given node's neighborhood and interprets it as a transaction database, which is used for frequent pattern mining to identify logical conjuncts of items that co-occur frequently in a given context. Such patterns ar…
Mining massive spatio-temporal data can help a variety of real-world applications such as city capacity planning, event management, and social network analysis. The tensor representation can be used to capture the correlation between space and time and simultaneously exploit the latent structure of the spatial and temporal patterns in an unsupervised fashion. However, the increasing volume of spatio-temporal data has made it prohibitively expensive to store and analyze using tensor factorization. In this paper, we propose SkeTenSmooth, a novel tensor factorization framework that uses adaptive …
