UQSay #80

The eightieth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, December 12, 2024.

2–3 PM — Claire Boyer (LMO, UPS) — [slides]

A primer on physics-informed learning

Physics-informed machine learning combines the expressiveness of data-based approaches with the interpretability of physical models. In this context, we consider a general regression problem where the empirical risk is regularized by a partial differential equation that quantifies the physical inconsistency. Practitioners often resort to physics-informed neural networks (PINNs) to solve this kind of problem. After discussing some strengths and limitations of PINNs, we prove that for linear differential priors, the problem can be formulated directly as a kernel regression task, giving a rigorous framework to analyze physics-informed ML. In particular, the physical prior can help in boosting the estimator convergence. We also propose the PIKL algorithm (PIKL for physics-informed kernel learning) as a numerical strategy to implement this kernel method.

References:

• Convergence and error analysis of PINNs, Bernoulli 2024
• Physics-informed machine learning as a kernel method, 2024
• Physics-informed kernel learning, 2024

Joint work with N. Doumèche (Sorbonne University) & G. Biau (Sorbonne University) & F. Bach (INRIA).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Vincent Chabridon (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Clément Gauchy (CEA), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Sébastien Petit (LNE), Emmanuel Vazquez (L2S), Xujia Zhu (L2S).

Coordinators: Sidonie Lefebvre (ONERA) & Xujia Zhu (L2S)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #79

The seventy-ninth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, November 28, 2024.

2–3 PM — Sam Allen (ETH Zürich, Seminar of Statistics) — [slides]

Evaluating forecasts using positive definite kernels

Probabilistic forecasts take the form of probability distributions over the set of possible outcomes. To assess the quality of a probabilistic forecast, proper scoring rules condense forecast performance into a single numerical value, with which competing forecasters can be ranked and compared. Many popular scoring rules can be expressed in terms of positive definite kernels. This allows us to leverage well-established results on kernels when evaluating probabilistic forecasts. In this work, we discuss two useful connections between kernel methods and scoring rules. Firstly, we demonstrate that well-known weighted scoring rules correspond to kernel scores. Weighted scoring rules allow users to emphasise particular outcomes during forecast evaluation, and by expressing these as kernel scores, we introduce a broad generalisation of existing weighted scores that can be applied to probabilistic forecasts on arbitrary outcome domains. This facilitates the introduction of weighted multivariate scoring rules, for example. Secondly, we demonstrate that by embedding probabilistic forecasts into a Reproducing Kernel Hilbert Space (RKHS), estimating the parameters of forecast models by optimising a kernel score often corresponds to a convex quadratic optimisation problem, which can therefore be solved robustly and efficiently. This encompasses common forecast combination methods, such as linear pooling, providing a means to optimally combine competing forecasts made on arbitrary outcome domains. To illustrate the practical benefit of these results, the two applications of kernel scores are applied in case studies on operational weather forecasts..

References:

• Evaluating Forecasts for High-Impact Events Using Transformed Kernel Scores, 2023
• Weighted scoringRules: Emphasizing Particular Outcomes When Evaluating Probabilistic Forecasts, 2024 - [R package]- [Python package]
• Efficient pooling of predictions via kernel embeddings, 2024

Joint work with D. Ginsbourger (University of Bern) & J. Ziegel (ETH Zürich).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Vincent Chabridon (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Clément Gauchy (CEA), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Sébastien Petit (LNE), Emmanuel Vazquez (L2S), Xujia Zhu (L2S).

Coordinators: Sidonie Lefebvre (ONERA) & Xujia Zhu (L2S)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #78

The seventy-eighth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, November 14, 2024.

2–3 PM — Botond Szabo (Bocconi University, Department of Decision Sciences - BIDSA) — [slides]

Uncertainty quantification and estimation with variational Gaussian Processes

We study the theoretical properties of a variational Bayes method in the Gaussian Process regression model. We consider the inducing variables method introduced by Titsias (2009b) and derive sufficient conditions for obtaining contraction rates for the corresponding variational Bayes (VB) posterior. We also derive guarantees and limitations for the associated credible sets. The derived general results are then applied for several specific examples. Finally, we also consider approximation methods from probabilistic numerics (e.g. Lanczos iteration and conjugate gradient descent). We demonstrate their close relationship with the variational Bayes approach and provide guidelines for these approaches as well to obtain optimal (minimax) statistical inference.

References:

• Contraction rates for sparse variational approximations in Gaussian process regression, 2022
• Uncertainty quantification for sparse spectral variational approximations in Gaussian process regression, 2023
• Contraction rates for conjugate gradient and Lanczos approximate posteriors in Gaussian process regression, 2024

Joint work with H. van Zanten (Vrije Universiteit Amsterdam) & D. Nieman (Vrije Universiteit Amsterdam) & B. Stankewitz (University of Potsdam).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Vincent Chabridon (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Clément Gauchy (CEA), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Sébastien Petit (LNE), Emmanuel Vazquez (L2S), Xujia Zhu (L2S).

Coordinators: Sidonie Lefebvre (ONERA) & Xujia Zhu (L2S)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #77

The seventy-seventh UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, October 31, 2024.

2–3 PM — Nadège Polette (Mines Paris PSL - CEA DAM DIF) — [slides]

Mitigating Overconfidence in Bayesian Field Inversion thanks to Hyperparameters Sampling

The objective of Bayesian field inversion is to approximate the posterior distribution of a field thanks to indirect observations. Such problems have to face two issues, the infinite dimensionality of the field and the high number of forward model evaluations required to achieve MCMC convergence. In this study, the field to infer is assumed to be a particular realization of a random field and is modeled by its truncated Karhunen-Loève (KL) decomposition leading to a finite dimensional parametrization. The KL representation relies on an autocovariance function that depends on poorly known hyperparameters. The added value of our work is to introduce a new method, called change of measure, designed to deal with uncertain hyperparameters instead of deterministic ones. In practice, the hyperparameters are jointly sampled with the KL coordinates and the posterior distribution has a hierarchical Bayesian structure. In addition, to reduce the computational cost of the MCMC procedure, the likelihood is estimated with polynomial chaos surrogates of the forward model outputs. Applications on transient diffusion and seismic traveltime tomography problems highlight the interest of not fixing the hyperparameters to deterministic values. Exploring the hyperparameters space is highly valuable because it provides a better estimation of the field uncertainties.

References:

• Change of Measure for Bayesian Field Inversion with Hierarchical Hyperparameters Sampling, 2024

Joint work with O. Le Maître (CNRS, CMAP) & P. Sochala (CEA DAM DIF) & A. Gesret (Mines Paris PSL).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Vincent Chabridon (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Clément Gauchy (CEA), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Sébastien Petit (LNE), Emmanuel Vazquez (L2S), Xujia Zhu (L2S).

Coordinators: Sidonie Lefebvre (ONERA) & Xujia Zhu (L2S)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #76

The seventy-sixth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, October 17, 2024.

4–5 PM — Habib Najm (Sandia National Laboratories) — [slides]

Uncertainty Quantification in Computational Combustion Models

Uncertainty quantification (UQ) in large scale computational combustion models faces key challenges of high dimensionality and computational cost. These models, particularly when using detailed chemical mechanisms, typically involve large numbers of uncertain parameters. Exploring these high-dimensional spaces necessitates the use of large numbers of computational samples, which, given high computational costs, is prohibitively expensive. I will discuss a UQ workflow, and underlying methods, to address this challenge in practice. These methods include global sensitivity analysis with polynomial chaos (PC) sparse regression, coupled with multilevel multifidelity methods. The combination of these tools is often useful to reliably cut-down dimensionality with acceptable computational costs, identifying a lower-dimensional subspace where the construction of PC surrogates of requisite accuracy is feasible. These surrogates are in turn necessary for both Bayesian inference and forward uncertainty propagation purposes. I will discuss this UQ workflow for problems of practical relevance in combustion. I will also touch on the issue of data availability, and the challenge of Bayesian estimation of uncertain model parameters given sparse or missing data.

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Vincent Chabridon (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Clément Gauchy (CEA), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Sébastien Petit (LNE), Emmanuel Vazquez (L2S), Xujia Zhu (L2S).

Coordinators: Sidonie Lefebvre (ONERA) & Xujia Zhu (L2S)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #75

The seventy-fifth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, October 3, 2024.

2–3 PM — Olivier Laurent (SATIE, Université Paris-Saclay - U2IS, ENSTA) — [slides]

A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors

The distribution of modern deep neural networks (DNNs) weights -- crucial for uncertainty quantification and robustness -- is an eminently complex object due to its extremely high dimensionality. This paper presents one of the first large-scale explorations of the posterior distribution of deep Bayesian Neural Networks (BNNs), expanding its study to real-world vision tasks and architectures. Specifically, we investigate the optimal approach for approximating the posterior, analyze the connection between posterior quality and uncertainty quantification, delve into the impact of modes on the posterior, and explore methods for visualizing the posterior. Moreover, we uncover weight-space symmetries as a critical aspect for understanding the posterior. To this extent, we develop an in-depth assessment of the impact of both permutation and scaling symmetries that tend to obfuscate the Bayesian posterior. While the first type of transformation is known for duplicating modes, we explore the relationship between the latter and L2 regularization, challenging previous misconceptions. Finally, to help the community improve our understanding of the Bayesian posterior, we release a large-scale dataset of model checkpoints, including thousands of real-world models, along with our code.

References:

• A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors, ICLR, 2024 - [webpage]- [code]
• TorchUncertainty package - [associated bdd]

Joint work with Emanuel Aldea (SATIE) & Gianni Franchi (U2IS, ENSTA).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Vincent Chabridon (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Clément Gauchy (CEA), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Sébastien Petit (LNE), Emmanuel Vazquez (L2S), Xujia Zhu (L2S).

Coordinators: Sidonie Lefebvre (ONERA) & Xujia Zhu (L2S)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #74

The seventy-fourth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, September 19, 2024.

3–4 PM — Alexander Terenin (Cornell University) — [slides]

Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent

The ability to deploy Gaussian-process-based decision-making systems such as Bayesian optimization at scale has traditionally been limited by computational costs arising from the need to solve large linear systems. The de facto standard for solving linear systems at scale is via the conjugate gradient algorithm - in particular, stochastic gradient descent is known to converge near-arbitrarily-slowly on quadratic objectives that correspond to Gaussian process models’ linear systems. In spite of this, we show that it produces solutions which have low test error, and quantify uncertainty in a manner that mirrors the true posterior. We develop a spectral characterization of the error caused by finite-time non-convergence, which we prove is small both near the data, and sufficiently far from the data. Stochastic gradient descent therefore only differs from the true posterior between these regions, demonstrating a form of implicit bias caused by benign non-convergence. We conclude by showing, empirically, that stochastic gradient descent achieves state-of-the-art performance on sufficiently large-scale regression tasks, and produces uncertainty estimates which match the performance of significantly more expensive baselines on large-scale Bayesian optimization.

References:

• Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent, NeurIPS, 2023 - [github sgd-gp]
• Stochastic Gradient Descent for Gaussian Processes Done Right, ICLR, 2024
• Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees, JMLR, 2024 - [github conjugate-gradient-sparse-gp]

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Vincent Chabridon (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Clément Gauchy (CEA), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Sébastien Petit (LNE), Emmanuel Vazquez (L2S), Xujia Zhu (L2S).

Coordinators: Sidonie Lefebvre (ONERA) & Xujia Zhu (L2S)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.