UQSay #72

The seventy-second UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, April 25, 2024.

3–4 PM — John Jakeman (Sandia National Laboratories)

Surrogate Modeling for Efficiently, Accurately and Conservatively Estimating Measures of Risk

This talk will present a surrogate-based framework for conservatively estimating risk from limited evaluations of an expensive physical experiment or simulation. Focus will be given to the computation of risk measures that quantify tail statistics of the loss, such as Average Value at Risk (AVaR). Monte Carlo (MC) sampling can be used to approximate such risk measures, however MC requires a large number of model simulations, which can make accurately estimating risk intractable for computationally expensive models. Given a set of samples surrogates are constructed such that the estimate of risk, obtained from the surrogate, is always greater than the empirical estimate obtained from the training data. These surrogates not only limit over-confidence in model reliability but produce estimates of risk that converge much faster to the true risk, than purely sampled based estimates.

The first part of the talk will discuss how to use the risk quadrangle, which rigorously connects stochastic optimization and statistical estimation, to construct conservative surrogates that can be tailored to the specific risk preferences of the model stakeholder. Surrogates constructed using least squares and quantile regression are specific cases of this framework. The second part of the talk will then present an approach, based upon stochastic orders, for constructing surrogates that are conservative with respect to families of risk measures, which is useful when risk preferences are difficult to elicit. This approach uses first and second order stochastic dominance to respectively enforce that the surrogate over-estimates probability of failure and AVaR for a finite set of thresholds. The conservative surrogates constructed introduce a bias that allows them to conservatively estimate risk. Theoretical results will be provided that show that for orthonormal models such as polynomial chaos expansions, this bias decays at the same rate as the root mean squared error in the surrogate. Numerous numerical examples will be used to confirm that that risk-aware surrogates do indeed overestimate risk while converging at the expected rate.

Reference: Surrogate modeling for efficiently, accurately and conservatively estimating measures of risk, RESS, 2022 - [github PyApprox].

Joint work with Drew Kouri (Sandia).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Emmanuel Vazquez (L2S).

Coordinators: Julien Bect (L2S) & Sidonie Lefebvre (ONERA)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #71

The seventy-first UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, March 21, 2024.

2–3 PM — Morgane Menz (IFPEN) — [slides]

Archissur: an Active Recovery of Constrained and Hidden Sets by SUR method. Coupling with adaptive space-filling and optimization

The analysis of simulated engineering systems (robust optimization, reliability assessment, …) generally requires numerous computationally expensive code simulations with different possible sets of values of design and environmental input variables. However, the simulators can encounter simulation crashes due to convergence issues for some values of both input variables. These failures correspond to a hidden constraint and might be as costly to evaluate as a feasible simulation. The presence of such crashes must be managed in a wise way, in order to target feasible input areas and thus avoid unnecessary irrelevant simulations.

In this context, we propose an adaptive strategy to learn the hidden constraint at a reduced numerical cost based only on a limited number of binary observations corresponding to failure or non-failure status. Our approach is a Gaussian Process Classifier active learning method based on Stepwise Uncertainty Reduction strategies to assess hidden constraints prediction. A numerically effective formulation of the enrichment criterion suited for classification is provided. Additionally, the proposed enrichment criterion is employed to address metamodeling and optimization in the presence of hidden constraints..

Reference: Estimation of simulation failure set with active learning based on Gaussian Process classifiers and random set theory, 2023.

Joint work with Miguel Munoz-Zuniga (IFPEN) & Delphine Sinoquet (IFPEN).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Emmanuel Vazquez (L2S).

Coordinators: Julien Bect (L2S) & Sidonie Lefebvre (ONERA)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #70

The seventieth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, March 7, 2024.

2–3 PM — Baptiste Kerleguer (CEA, DAM/DIF) — [slides]

A Bayesian neural network approach to multi-fidelity surrogate modeling

This talk deals with the surrogate modeling of computer code results that can be evaluated at different levels of accuracy and computational cost, called multi-fidelity. We propose a method combining Gaussian process (GP) regression on low-fidelity data and a Bayesian neural network (BNN) on high-fidelity data. The novelty, compared with the state of the art, is that uncertainties are taken into account at all fidelity levels. The prediction uncertainty of the low-fidelity level is transmitted by Gauss-Hermite quadrature to the high-fidelity level. In addition, this method takes into account non-nested designs of experiment and non-linear interactions between levels. The proposed approach is then compared to several multi-fidelity GP regression methods on analytic functions and on a computer code.

Reference: A Bayesian neural network approach to multi-fidelity surrogate modeling, IJUQ 14.1, 2024.

Joint work with Josselin Garnier (CMAP) & Claire Cannamela (CEA, DAM/DIF).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Emmanuel Vazquez (L2S).

Coordinators: Julien Bect (L2S) & Sidonie Lefebvre (ONERA)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #69

The sixty-ninth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, February 8, 2024.

2–3 PM — Marouane Il Idrissi (EDF R&D - IMT) — [slides]

Generalized Hoeffding decomposition and the linear nature of non-linearity

Hoeffding’s decomposition of random outputs traditionally requires the inputs to be mutually independent. It allows uniquely decomposing a square-integrable function as a sum taken over every subset of inputs. Generalizing this result to non-mutually independent inputs has been a recent challenge in the literature on sensitivity analysis. Proposed solutions exist, but they require relatively restrictive assumptions on the distribution of the inputs. However, Hoeffding’s decomposition can be generalized under two reasonable assumptions on the inputs’ distribution: non-perfect functional dependence and non-degenerate stochastic dependence.

This generalization requires approaching the problem using a framework at the cornerstone of probability theory, functional analysis, and combinatorics. From this perspective, it can be seen as finding a direct-sum decomposition of a particular Lebesgue space, unveiling a surprisingly linear approach to handling stochastic and functional non-linearities. The proposed "ortho-canonical decomposition" relies on oblique projections rather than the traditional conditional expectations. Ultimately, it allows the definition of intuitive and interpretable sensitivity indices, which offers a path toward a more precise uncertainty quantification.

In this talk, we will delve into the unconventional framework used, discuss its nuances, and explore the various perspectives and challenges it offers.

Reference: Understanding black-box models with dependent inputs through a generalization of Hoeffding's decomposition, 2023 [github].

Joint work with Nicolas Bousquet (EDF R&D - LPSM), Fabrice Gamboa (IMT), Bertrand Ioss (EDF R&D - IMT) and Jean-Michel Loubes (IMT).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Emmanuel Vazquez (L2S).

Coordinators: Julien Bect (L2S) & Sidonie Lefebvre (ONERA)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #68

The sixty-eighth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, January 25, 2024.

2–3 PM — Gabriel Sarazin (CEA Paris Saclay) — [slides]

Towards more interpretable kernel-based sensitivity analysis

When working with a computationally-expensive simulation code involving a large number of uncertain physical parameters, it is often advisable to perform a preliminary sensitivity analysis in order to identify which input variables will really be useful for surrogate modelling. On paper, the total-order Sobol' indices fulfill this role perfectly, since they are able to detect any type of input-output dependence, while being interpretable as simple percentages of the output variance. However, in many situations, their accurate estimation remains a thorny issue, despite remarkable progress in that direction over the past few months. In this context where inference is strongly constrained, kernel methods have emerged as an excellent alternative, notably through the Hilbert-Schmidt independence criterion (HSIC). Although they offer undeniable advantages over Sobol' indices, HSIC indices are much harder to understand, and this lack of interpretability is a major obstacle to their wider dissemination. In order to marry the advantages of Sobol' and HSIC indices, an ANOVA-like decomposition allows to define HSIC-ANOVA indices at all orders, just as would be done for Sobol' indices. This recent contribution is the starting point of this presentation.

The main objective of this talk is to provide deeper insights into the HSIC-ANOVA framework. One major difference with the basic HSIC framework lies in the use of specific input kernels (like Sobolev kernels). First, a dive into the universe of cross-covariance operators will allow to better understand how sensitivity is measured by HSIC-ANOVA indices, and what type of input-output dependence is captured by each term of the HSIC-ANOVA decomposition. Then, a brief study of Sobolev kernels, focusing more particularly on their feature maps, will reveal what kind of simulators are likely to elicit HSIC-ANOVA interactions. It will also be demonstrated that Sobolev kernels are characteristic, which ensures that HSIC-ANOVA indices can be used to test input-output independence. Finally, a test procedure will be proposed for the total-order HSIC-ANOVA index, and it will be shown (numerically) that the resulting test of independence is at least as powerful as the standard test (based on two Gaussian kernels).

Reference: New insights into the feature maps of Sobolev kernels: application in global sensitivity analysis, 2023 [sensiHSIC & testHSIC in R package sensitivity].

Joint work with Amandine Marrel (CEA Cadarache), Sébastien Da Veiga (ENSAI) and Vincent Chabridon (EDF R&D).

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Emmanuel Vazquez (L2S).

Coordinators: Julien Bect (L2S) & Sidonie Lefebvre (ONERA)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #67

The sixty-seventh UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, December 14, 2023.

2–3 PM — Pierre Humbert (LMO - INRIA) — [slides]

One-Shot Federated Conformal Prediction

In this presentation, we will focus on a method for constructing prediction sets in a federated learning setting where only one round of communication between the agents and the server is allowed (one-shot). More precisely, by defining a particular estimator called the quantile-of-quantiles, we will prove that for any distribution, it is possible to produce marginally (and training-conditionally) valid prediction sets. Over a wide range of experiments, we will show that we are able to obtain prediction sets whose coverage and length are very similar to those obtained in a centralized setting, making our method particularly well-suited to perform conformal predictions in a one-shot federated learning setting.

Reference: One-Shot Federated Conformal Prediction, ICML 2023

Joint work with Batiste Le Bars, Aurélien Bellet and Sylvain Arlot.

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Emmanuel Vazquez (L2S).

Coordinators: Julien Bect (L2S) & Sidonie Lefebvre (ONERA)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.

UQSay #66

The sixty-sixth UQSay seminar on UQ, DACE and related topics will take place online on Thursday afternoon, November 30, 2023.

2–3 PM — Elaine Spiller (Marquette University) — [slides]

Two recent advances in UQ with Gaussian process models:
the zGP and the PPLE

Gaussian processes (GPs) are an effective and widely used tools to emulate computer simulations of physical process models for uncertainty quantification (UQ). Over the last 10-15 years, GP modeling of computer simulations has advanced tremendously to handle challenges posed by complex and realistic simulators. We will discuss two recent challenges. The first challenge is the "zero-problem” — simulations that result in positive, real-valued output or zero. Such zero-censored data pose a significant obstacle to GP emulators because of both the inherent non-stationary and because GPs have full support. The second challenge we will explore is emulating high-dimensional multi-physics simulations. Here we will combine two recent GP approaches: linked GP emulation (for coupled physical simulations) and parallel partial emulators (PPEs) for emulating simulators with high-dimensional output. The resulting parallel partial linked GP emulator (PPLE) proves an efficient approach to emulate high-dimensional multi-physics simulators.

Reference: E.T. Spiller, R.W. Wolpert, P. Tierz & T.G. Asher, “The zero problem: Gaussian process emulators for range constrained computer models”, 2023. [github]

Joint work with Robert Wolpert (Duke Univ.) and Sue Minkoff (Univ. of Texas)

Organizing committee: Pierre Barbillon (MIA-Paris), Julien Bect (L2S), Nicolas Bousquet (EDF R&D), Amélie Fau (LMPS), Filippo Gatti (LMPS), Bertrand Iooss (EDF R&D), Alexandre Janon (LMO), Sidonie Lefebvre (ONERA), Didier Lucor (LISN), Emmanuel Vazquez (L2S).

Coordinators: Julien Bect (L2S) & Sidonie Lefebvre (ONERA)

Practical details: the seminar will be held online using Microsoft Teams.

If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).

You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.

The technical side of things: you can use Teams either directly from your web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.