Hawkes Processes Modeling, Inference and Control: An Overview

Mean-field inference of Hawkes point processes

We propose a fast and efficient estimation method that is able to accurately recover the parameters of a d-dimensional Hawkes point-process from a set of observations. We exploit a mean-field approximation that is valid when the fluctuations of the stochastic intensity are small. We show that this is notably the case in situations when interactions are sufficiently weak, when the dimension of the system is high or when the fluctuations are self-averaging due to the large number of past events they involve. In such a regime the estimation of a Hawkes process can be mapped on a least-squares pro…

Nonlinear Hawkes Process with Gaussian Process Self Effects

Traditionally, Hawkes processes are used to model time--continuous point processes with history dependence. Here we propose an extended model where the self--effects are of both excitatory and inhibitory type and follow a Gaussian Process. Whereas previous work either relies on a less flexible parameterization of the model, or requires a large amount of data, our formulation allows for both a flexible model and learning when data are scarce. We continue the line of work of Bayesian inference for Hawkes processes, and our approach dispenses with the necessity of estimating a branching structure…

Variational Inference for Sparse Gaussian Process Modulated Hawkes Process

The Hawkes process (HP) has been widely applied to modeling self-exciting events including neuron spikes, earthquakes and tweets. To avoid designing parametric triggering kernel and to be able to quantify the prediction confidence, the non-parametric Bayesian HP has been proposed. However, the inference of such models suffers from unscalability or slow convergence. In this paper, we aim to solve both problems. Specifically, first, we propose a new non-parametric Bayesian HP in which the triggering kernel is modeled as a squared sparse Gaussian process. Then, we propose a novel variational infe…

Cox-Hawkes: doubly stochastic spatiotemporal Poisson processes

Hawkes processes are point process models that have been used to capture self-excitatory behavior in social interactions, neural activity, earthquakes and viral epidemics. They can model the occurrence of the times and locations of events. Here we develop a new class of spatiotemporal Hawkes processes that can capture both triggering and clustering behavior and we provide an efficient method for performing inference. We use a log-Gaussian Cox process (LGCP) as prior for the background rate of the Hawkes process which gives arbitrary flexibility to capture a wide range of underlying background …

Uncertainty Quantification for Inferring Hawkes Networks

Multivariate Hawkes processes are commonly used to model streaming networked event data in a wide variety of applications. However, it remains a challenge to extract reliable inference from complex datasets with uncertainty quantification. Aiming towards this, we develop a statistical inference framework to learn causal relationships between nodes from networked data, where the underlying directed graph implies Granger causality. We provide uncertainty quantification for the maximum likelihood estimate of the network multivariate Hawkes process by providing a non-asymptotic confidence set. The…

FaDIn：基于一般参数化核的Hawkes过程的快速离散化推断

FaDIn: Fast Discretized Inference for Hawkes Processes with General Parametric Kernels

本文提出了一种基于一般参数化核和事件离散化的快速梯度求解器FaDIn，用于高效进行Hawkes过程推断，在脑磁图信号建模中实现了优于现有技术的模式延迟估计。

Modeling of Spatio-Temporal Hawkes Processes with Randomized Kernels

We investigate spatio-temporal event analysis using point processes. Inferring the dynamics of event sequences spatiotemporally has many practical applications including crime prediction, social media analysis, and traffic forecasting. In particular, we focus on spatio-temporal Hawkes processes that are commonly used due to their capability to capture excitations between event occurrences. We introduce a novel inference framework based on randomized transformations and gradient descent to learn the process. We replace the spatial kernel calculations by randomized Fourier feature-based transfor…

Efficient Inference of Flexible Interaction in Spiking-neuron Networks

Hawkes process provides an effective statistical framework for analyzing the time-dependent interaction of neuronal spiking activities. Although utilized in many real applications, the classic Hawkes process is incapable of modelling inhibitory interactions among neurons. Instead, the nonlinear Hawkes process allows for a more flexible influence pattern with excitatory or inhibitory interactions. In this paper, three sets of auxiliary latent variables (Pólya-Gamma variables, latent marked Poisson processes and sparsity variables) are augmented to make functional connection weights in a Gaussia…

Hawkes Processes with Stochastic Excitations

We propose an extension to Hawkes processes by treating the levels of self-excitation as a stochastic differential equation. Our new point process allows better approximation in application domains where events and intensities accelerate each other with correlated levels of contagion. We generalize a recent algorithm for simulating draws from Hawkes processes whose levels of excitation are stochastic processes, and propose a hybrid Markov chain Monte Carlo approach for model fitting. Our sampling procedure scales linearly with the number of required events and does not require stationarity of …

A Dirichlet Mixture Model of Hawkes Processes for Event Sequence Clustering

We propose an effective method to solve the event sequence clustering problems based on a novel Dirichlet mixture model of a special but significant type of point processes --- Hawkes process. In this model, each event sequence belonging to a cluster is generated via the same Hawkes process with specific parameters, and different clusters correspond to different Hawkes processes. The prior distribution of the Hawkes processes is controlled via a Dirichlet distribution. We learn the model via a maximum likelihood estimator (MLE) and propose an effective variational Bayesian inference algorithm.…

Can a latent Hawkes process be used for epidemiological modelling?

Understanding the spread of COVID-19 has been the subject of numerous studies, highlighting the significance of reliable epidemic models. Here, we introduce a novel epidemic model using a latent Hawkes process with temporal covariates for modelling the infections. Unlike other models, we model the reported cases via a probability distribution driven by the underlying Hawkes process. Modelling the infections via a Hawkes process allows us to estimate by whom an infected individual was infected. We propose a Kernel Density Particle Filter (KDPF) for inference of both latent cases and reproductio…

Statistical Inference for Networks of High-Dimensional Point Processes

Fueled in part by recent applications in neuroscience, the multivariate Hawkes process has become a popular tool for modeling the network of interactions among high-dimensional point process data. While evaluating the uncertainty of the network estimates is critical in scientific applications, existing methodological and theoretical work has primarily addressed estimation. To bridge this gap, this paper develops a new statistical inference procedure for high-dimensional Hawkes processes. The key ingredient for this inference procedure is a new concentration inequality on the first- and second-…

Hawkes过程的共轭贝叶斯两步变点检测

Conjugate Bayesian Two-step Change Point Detection for Hawkes Process

针对Hawkes过程中传统贝叶斯两步变点检测因非共轭性导致计算效率低的问题，本文利用数据增强提出一种共轭贝叶斯方法，实现了更准确高效的变点检测。

具有延迟Granger因果关系的Hawkes过程

Hawkes Processes with Delayed Granger Causality

本文提出在多元Hawkes过程中显式建模延迟Granger因果效应，证明延迟参数可识别，并用VAE推断时间滞后后验分布，在合成与真实数据上验证了事件预测和滞后推断的有效性。

Causal Discovery in Hawkes Processes by Minimum Description Length

Hawkes processes are a special class of temporal point processes which exhibit a natural notion of causality, as occurrence of events in the past may increase the probability of events in the future. Discovery of the underlying influence network among the dimensions of multi-dimensional temporal processes is of high importance in disciplines where a high-frequency data is to model, e.g. in financial data or in seismological data. This paper approaches the problem of learning Granger-causal network in multi-dimensional Hawkes processes. We formulate this problem as a model selection task in whi…

可重构硬件上卷积神经网络推理的算术适配综述

An Overview of Arithmetic Adaptations for Inference of Convolutional Neural Networks on Re-configurable Hardware

本文针对卷积神经网络在FPGA等可重构嵌入式硬件部署的难点，介绍了算术适配相关策略，并展示了基于TinyYOLOv3在XILINX Artix-7 FPGA上结合批归一化融合等技术的最佳实践。

通过对比关系推理增强异步时间序列预测

Enhancing Asynchronous Time Series Forecasting with Contrastive Relational Inference

针对现有TPP模型未显式建模事件交互的问题，提出基于对比关系推理的霍克斯过程CRIHP，结合NRI与变分推理，通过强度学习对比关系约束，在三个真实数据集上验证了捕捉事件交互的有效性，代码将集成入EasyTPP。

基于基础推理模型的时间点过程上下文学习

In-Context Learning of Temporal Point Processes with Foundation Inference Models

提出一种基于摊销推理和上下文学习的点过程基础推理模型FIM-PP，通过大规模合成数据预训练，无需额外训练即可估计真实MTPP，或快速微调至目标系统。

基于核嵌入的因果推断综述

An Overview of Causal Inference using Kernel Embeddings

本文综述了核嵌入方法在因果推断领域的应用，介绍其通过将概率测度映射到RKHS实现灵活分布表示，为解决观测数据的因果识别与效应估计等问题提供鲁棒非参数框架。

An Overview of Uncertainty Quantification Methods for Infinite Neural Networks

To better understand the theoretical behavior of large neural networks, several works have analyzed the case where a network's width tends to infinity. In this regime, the effect of random initialization and the process of training a neural network can be formally expressed with analytical tools like Gaussian processes and neural tangent kernels. In this paper, we review methods for quantifying uncertainty in such infinite-width neural networks and compare their relationship to Gaussian processes in the Bayesian inference framework. We make use of several equivalence results along the way to o…

面向数据自适应序列贝叶斯推理的可微粒子滤波器综述

An overview of differentiable particle filters for data-adaptive sequential Bayesian inference

本文综述了可微粒子滤波器的最新进展与应用，重点梳理其动态模型、测量模型、提议分布等关键组件的不同设计选择，这类方法用神经网络构建PF组件并梯度优化，可适配复杂高维序列推理任务。

内存即一切：用于加速大语言模型推理的存内计算架构综述

Memory Is All You Need: An Overview of Compute-in-Memory Architectures for Accelerating Large Language Model Inference

本文综述面向大语言模型推理加速的存内计算架构，分析 transformer 算子、硬件方案及内存墙等挑战。

Adversary-resilient Distributed and Decentralized Statistical Inference and Machine Learning: An Overview of Recent Advances Under the Byzantine Threat Model

While the last few decades have witnessed a huge body of work devoted to inference and learning in distributed and decentralized setups, much of this work assumes a non-adversarial setting in which individual nodes---apart from occasional statistical failures---operate as intended within the algorithmic framework. In recent years, however, cybersecurity threats from malicious non-state actors and rogue entities have forced practitioners and researchers to rethink the robustness of distributed and decentralized algorithms against adversarial attacks. As a result, we now have a plethora of algor…

具有事件时间不确定性的点过程

Point processes with event time uncertainty

本文提出了一种处理时间不确定性的Hawkes过程建模框架，通过离散时间模型和优化方法提升推断效率，并在多个数据集上验证了其优越性。

Scalable Marked Point Processes for Exchangeable and Non-Exchangeable Event Sequences

We adopt the interpretability offered by a parametric, Hawkes-process-inspired conditional probability mass function for the marks and apply variational inference techniques to derive a general and scalable inferential framework for marked point processes. The framework can handle both exchangeable and non-exchangeable event sequences with minimal tuning and without any pre-training. This contrasts with many parametric and non-parametric state-of-the-art methods that typically require pre-training and/or careful tuning, and can only handle exchangeable event sequences. The framework's competit…

Security for Machine Learning-based Systems: Attacks and Challenges during Training and Inference

The exponential increase in dependencies between the cyber and physical world leads to an enormous amount of data which must be efficiently processed and stored. Therefore, computing paradigms are evolving towards machine learning (ML)-based systems because of their ability to efficiently and accurately process the enormous amount of data. Although ML-based solutions address the efficient computing requirements of big data, they introduce (new) security vulnerabilities into the systems, which cannot be addressed by traditional monitoring-based security measures. Therefore, this paper first pre…

Advances in Variational Inference

Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully used in various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent ad…

Modeling the Dynamics of Online Learning Activity

People are increasingly relying on the Web and social media to find solutions to their problems in a wide range of domains. In this online setting, closely related problems often lead to the same characteristic learning pattern, in which people sharing these problems visit related pieces of information, perform almost identical queries or, more generally, take a series of similar actions. In this paper, we introduce a novel modeling framework for clustering continuous-time grouped streaming data, the hierarchical Dirichlet Hawkes process (HDHP), which allows us to automatically uncover a wide …

The Block Point Process Model for Continuous-Time Event-Based Dynamic Networks

We consider the problem of analyzing timestamped relational events between a set of entities, such as messages between users of an on-line social network. Such data are often analyzed using static or discrete-time network models, which discard a significant amount of information by aggregating events over time to form network snapshots. In this paper, we introduce a block point process model (BPPM) for continuous-time event-based dynamic networks. The BPPM is inspired by the well-known stochastic block model (SBM) for static networks. We show that networks generated by the BPPM follow an SBM i…

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review

The framework of reinforcement learning or optimal control provides a mathematical formalization of intelligent decision making that is powerful and broadly applicable. While the general form of the reinforcement learning problem enables effective reasoning about uncertainty, the connection between reinforcement learning and inference in probabilistic models is not immediately obvious. However, such a connection has considerable value when it comes to algorithm design: formalizing a problem as probabilistic inference in principle allows us to bring to bear a wide array of approximate inference…