Seminar on Statistics and Data Science

ICA-based causal discovery methods such as LiNGAM have been highly successful under the assumption that noise variables become independent after an appropriate causal ordering. However, this assumption is often violated in the presence of confounding. \[ \] In this talk, I present a Bayesian and information-theoretic formulation of ICA for causal order estimation that explicitly allows for confounding. Rather than enforcing independence, we quantify residual dependence among noise variables using multivariate mutual information and evaluate causal orders via Bayesian marginal likelihoods. \[ \] This approach provides a principled ranking of causal orders under confounding and recovers classical LiNGAM-type methods as special cases when confounding is absent. I will focus on the conceptual framework and discuss connections to existing ICA-based methods, as well as open questions.

Suppose we are given some data, and we hypothesize a structural causal model to describe them: how can we narrow the set of causal graphs compatible with our observations? The theory of identifiability aims to answer this question. We show that, in the case of additive noise models, the score function of the data contains all the information about the causal graph. However, this requires strong and, crucially, hard-to-verify modeling assumptions, like additivity of the noise. When direct experiments to infer causality are not feasible, this raises the question: how can we move past these restrictions? Borrowing ideas from independent component analysis, we show how multiple environments (read: non i.i.d. data) can overcome these limitations: for structural causal models with arbitrary causal mechanisms, data from only three environments uniquely identify the causal graph from the Jacobian of the score function. Thus, non-i.i.d.-ness turns from a curse into a blessing for causal discovery.

We consider the task of learning dynamical systems from data in a system-agnostic framework. This presentation is divided into two parts. First, we utilize a simulation-based approach to investigate algorithms for learning chaotic Ordinary Differential Equations (ODEs). We demonstrate that for noise-free data and low-dimensional systems, this task is effectively solved, as polynomial regression-based methods can achieve machine-precision forecasts. However, we show that observational noise remains a significant challenge for most algorithms. In the second part, we address this challenge by developing nonparametric statistical theory for learning ODEs from noisy observations. Specifically, we establish minimax optimal error rates for two contrasting observational models: the Stubble model, consisting of many short trajectories, and the Snake model, consisting of a single long trajectory. We conclude by discussing challenges at the intersection of dynamical systems and statistical learning.

Consider the regression problem where the response and the covariate are unmatched. Under this scenario, we do not have access to pairs of observations from their joint distribution, but instead we have separate data sets of responses and covariates, possibly collected from different sources. We study this problem assuming that the regression function is linear and the noise distribution is known or can be estimated. We introduce an estimator of the regression vector based on deconvolution (the DLSE) and demonstrate its consistency and asymptotic normality under parametric identifiability. Under non-identifiability of the regression vector but identifiability of the distribution of the predictor, we construct an estimator of the latter based on the DLSE and show that it converges to the true distribution of the predictor at the parametric rate in the Wasserstein distance of order 1. We illustrate the theory with several simulation results. \[ \] This talk is based on my joint work with Mona Azadkia, Antonio di Noia and Cecile Durot

Real-life financial time series exhibit heavy tails and clusters of extreme values. In this talk we will address models that exhibit these stylized facts. This is the class of regularly varying time series, introduced by Davis and Hsing (1995, AoP) and further developed by Basrak and Segers (2009, SPA). The marginal distribution of a regularly varying time series has tails of power-law type, and the dynamics caused by an extreme event in this time series is described by the spectral tail process. The perhaps best known financial time series models of this kind are Engle’s (1982) ARCH process, Bollerslev’s (1986) GARCH process and Engle’s and Russell’s (1998) Autoregressive Conditional Duration (ACD) model. The length and magnitude of extremal clusters in such a series can be described by an analog of the autocorrelation function for extreme events: the extremogram. The extremal index is another useful tool for describing expected extremal cluster sizes. Both objects can be expressed in terms of the spectral tail process and allow for statistical estimation. The probabilistic and statistical aspects of regularly varying time series are summarized in the recent monograph by Mikosch and Wintenberger (2024) “Extreme Value Theory for Time Series. Models with Power-Law Tails”. The talk is based on joint work with Olivier Wintenberger (Sorbonne).

Diffusion models are one of the key architectures of generative AI. Their main drawback, however, is the high computational cost. The first part of the talk introduces diffusion models, outlining their main ideas and their role in modern generative AI. The second part of the talk will focus on our recent research showing how the concept of sparsity, well known especially in statistics, can provide a pathway to more efficient diffusion pipelines. Our mathematical guarantees prove that sparsity can reduce the influence of the input dimension on computational complexity to that of a much smaller intrinsic dimension of the data. Our empirical findings further confirm that inducing sparsity can indeed lead to better samples at a lower cost.

The talk is divided into two parts. In the first part, I will introduce structural equation processes as a model for causal inference in discrete-time stationary processes. A structural equation process (SEP) consists of a directed graph, an independent stationary (zero-mean) process for every vertex of the graph, and a filter (i.e., an absolutely summable sequence) for every link on the graph. Every structural vector autoregressive (SVAR) process, a commonly used linear time series model, admits a representation as a SEP. Furthermore, the Fourier-transformed SEP representation of an SVAR process is parameterized over the field of rational functions with real coefficients. Using this frequency domain parameterization, we will see that d- and t- separation statements about the causal graph (associated with the SVAR process) are generically characterized by rank conditions on the spectral density of the SVAR process. Here, the spectral density is considered as a matrix over the field of rational functions with real coefficients. Additionally, we will see that the Fourier-transformed SEP parameterization of an SVAR process comes with a notion of rational identifiability for the Fourier transformed link filters. This notion allows to reason about identifiability in the presence of latent confounding processes. For instance, the recent latent factor half-trek criterion can be used to determine if the effect (i.e., the associated link function) between two potentially confounded processes is a rational function of the spectral density of the observed processes. \[ \] In the second part of the talk, I will expand the SEP framework to include a specific class of non-stationary linear processes. This class of non-stationary SEPs includes SVAR processes with periodically changing coefficients. I will also demonstrate how this framework can be used to reason about identifiability in subsampled processes, i.e., when observations are gathered at a lower frequency than the frequency at which causal effects occur.