Seminar on Statistics and Data Science

t.b.a.

t.b.a.

t.b.a.

We extend the scope of a recently introduced dependence coefficient between a scalar response Y and a multivariate covariate X to the case where X takes values in a general metric space. Particular attention is paid to the case where X is a curve. While on the population level, this extension is straight forward, the asymptotic behavior of the estimator we consider is delicate. It crucially depends on the nearest neighbor structure of the infinite-dimensional covariate sample, where deterministic bounds on the degrees of the nearest neighbor graphs available in multivariate settings do no longer exist. The main contribution of this paper is to give some insight into this matter and to advise a way how to overcome the problem for our purposes. As an important application of our results, we consider an independence test.

t.b.a.

We consider the problem of making nonparametric inference in multi-dimensional diffusion models from low-frequency data. Statistical analysis in this setting is notoriously challenging due to the intractability of the likelihood and its gradient, and computational methods have thus far largely resorted to expensive simulation-based techniques. In this article, we propose a new computational approach which is motivated by PDE theory and is built around the characterisation of the transition densities as solutions of the associated heat (Fokker-Planck) equation. Employing optimal regularity results from the theory of parabolic PDEs, we prove a novel characterisation for the gradient of the likelihood. Using these developments, for the nonlinear inverse problem of recovering the diffusivity (in divergence form models), we then show that the numerical evaluation of the likelihood and its gradient can be reduced to standard elliptic eigenvalue problems, solvable by powerful finite element methods. This enables the efficient implementation of a large class of statistical algorithms, including (i) preconditioned Crank-Nicolson and Langevin-type methods for posterior sampling, and (ii) gradient-based descent optimisation schemes to compute maximum likelihood and maximum-a-posteriori estimates. We showcase the effectiveness of these methods via extensive simulation studies in a nonparametric Bayesian model with Gaussian process priors. Interestingly, the optimisation schemes provided satisfactory numerical recovery while exhibiting rapid convergence towards stationary points despite the problem nonlinearity; thus our approach may lead to significant computational speed-ups.

This short course covers recent developments in graphical and causal modeling in Statistics/Machine Learning. It is comprised of the following three lectures, each two hours long. \[ \] June 25, 2024; Lecture 1: “Learning from conditional independence when not all variables are measured: Ancestral graphs and the FCI algorithm” \[ \] June 27, 2024; Lecture 2: “Identification of causal effects: A reformulation of the ID algorithm via the fixing operation” \[ \] July 2, 2024; Lecture 3: “Nested Markov models” \[ \] The course targets an audience with exposure to basic concepts in graphical and causal modeling (e.g., conditional independence, DAGs, d-separation, Markov equivalence, definition of causal effects/the do-operator).

This short course covers recent developments in graphical and causal modeling in Statistics/Machine Learning. It is comprised of the following three lectures, each two hours long. \[ \] June 25, 2024; Lecture 1: “Learning from conditional independence when not all variables are measured: Ancestral graphs and the FCI algorithm” \[ \] June 27, 2024; Lecture 2: “Identification of causal effects: A reformulation of the ID algorithm via the fixing operation” \[ \] July 2, 2024; Lecture 3: “Nested Markov models” \[ \] The course targets an audience with exposure to basic concepts in graphical and causal modeling (e.g., conditional independence, DAGs, d-separation, Markov equivalence, definition of causal effects/the do-operator).

This short course covers recent developments in graphical and causal modeling in Statistics/Machine Learning. It is comprised of the following three lectures, each two hours long. \[ \] June 25, 2024; Lecture 1: “Learning from conditional independence when not all variables are measured: Ancestral graphs and the FCI algorithm” \[ \] June 27, 2024; Lecture 2: “Identification of causal effects: A reformulation of the ID algorithm via the fixing operation” \[ \] July 2, 2024; Lecture 3: “Nested Markov models” \[ \] The course targets an audience with exposure to basic concepts in graphical and causal modeling (e.g., conditional independence, DAGs, d-separation, Markov equivalence, definition of causal effects/the do-operator).

The ultimate goal of causal inference is so-called causal effect identification (ID), which refers to quantifying the causal influence of a subset of variables on a target set. A stepping stone towards performing ID is learning the causal relationships among the variables which is commonly called causal structure learning (CSL). In this talk, I mainly focus on the problems pertaining to CSL and ID in linear structural causal models, which serve as the basis for problem abstraction in various scientific fields. In particular, I will review the identifiability results and algorithms for CSL and ID in the presence of latent confounding. Then, I will present our recent result on the ID problem using cross-moments among observed variables and discuss its applications to natural experiments and proximal causal inference. Finally, I conclude the presentation with possible future research directions.

One pervasive task found throughout the empirical sciences is to determine the effect of interventions from observational (non-experimental) data. It is well-understood that assumptions are necessary to perform causal inferences, which are commonly articulated through causal diagrams (Pearl, 2000). Despite the power of this approach, there are settings where the knowledge necessary to fully specify a causal diagram may not be available, particularly in complex, high-dimensional domains. In this talk, I will briefly present two recent causal effect identification results that relax the stringent requirement of fully specifying a causal diagram. The first is a new graphical modeling tool called cluster DAGs (for short, C-DAGs) that allows for the specification of relationships among clusters of variables, while the relationships between the variables within a cluster are left unspecified [1]. The second includes a complete calculus and algorithm for effect identification from a Partial Ancestral Graph (PAG), which represents a Markov equivalence class of causal diagrams, fully learnable from observational data [2]. These approaches are expected to help researchers and data scientists to identify novel effects in real-world domains, where knowledge is largely unavailable and coarse. \[ \] References: [1] Anand, T. V., Ribeiro, A. H., Tian, J., & Bareinboim, E. (2023). Causal Effect Identification in Cluster DAGs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, No. 10, pp. 12172-12179. [2] Jaber, A., Ribeiro, A., Zhang, J., & Bareinboim, E. (2022). Causal identification under markov equivalence: Calculus, algorithm, and completeness. Advances in Neural Information Processing Systems, 35, 3679-3690.

Forecast reconciliation has attracted significant research interest in recent years, with most studies taking the hierarchy of time series as given. We extend existing work that uses time series clustering to construct hierarchies, with the goal of improving forecast accuracy. First, we investigate multiple approaches to clustering, including not only different clustering algorithms, but also the way time series are represented and how distance between time series is defined. Second, we devise an approach based on random permutation of hierarchies, keeping the structure of the hierarchy fixed, while time series are randomly allocated to clusters. Third, we propose an approach based on averaging forecasts across hierarchies constructed using different clustering methods, that is shown to outperform any single clustering method. Our findings provide new insights into the role of hierarchy construction in forecast reconciliation and offer valuable guidance on forecasting practice.

In the context of linear regression, we construct a data-driven convex loss function with respect to which empirical risk minimisation yields optimal asymptotic variance in the downstream estimation of the regression coefficients. Our semiparametric approach targets the best decreasing approximation of the derivative of the log-density of the noise distribution. At the population level, this fitting process is a nonparametric extension of score matching, corresponding to a log-concave projection of the noise distribution with respect to the Fisher divergence. The procedure is computationally efficient, and we prove that our procedure attains the minimal asymptotic covariance among all convex M-estimators. As an example of a non-log-concave setting, for Cauchy errors, the optimal convex loss function is Huber-like, and our procedure yields an asymptotic efficiency greater than 0.87 relative to the oracle maximum likelihood estimator of the regression coefficients that uses knowledge of this error distribution; in this sense, we obtain robustness without sacrificing much efficiency.

When learning a causal model of a system, a key motivation is the use of that model for downstream decision-making. In this talk, I will take a decision-centric perspective on causal structure learning, focused on a simple setting that is amenable to careful statistical analysis. In particular, we study causal effect estimation via covariate adjustment, when the causal graph is unknown, all variables are discrete, and the non-descendants of treatment are given. \[ \] We propose an algorithm which searches for a data-dependent "approximate" adjustment set via conditional independence testing, and analyze the bias-variance tradeoff entailed by this procedure. We prove matching upper and lower bounds on omitted confounding bias in terms of small violations of conditional independence. Further, we provide a finite-sample bound on the complexity of correctly selecting an "approximate" adjustment set and of estimating the resulting adjustment functional, using results from the property testing literature. \[ \] We demonstrate our algorithm on synthetic and real-world data, outperforming methods which ignore structure learning or which perform structure learning separately from causal effect estimation. I conclude with some open questions at the intersection of structure learning and causal effect estimation.