The twenty-fifth UQSay seminar on UQ, DACE and related topics, organized by L2S, MSSMAT, LMT and EDF R&D, will take place online on Thursday afternoon, March 4, 2021.
2–3 PM — Victor Picheny (Secondmind) — [slides]
Bayesian optimisation: ablation study, global performance assessment and improvements based on trust regions
Bayesian Optimisation algorithms (BO) are global optimisation methods that iterate by constructing and using conditional Gaussian processes (GP). It is a common claim that BO is state-of-the-art for costly functions. However, this claim is weakly supported by experimental evidence, as BO is most often compared to itself, rather than to algorithms of different nature.
In this work, we study the performance of BO within the well-known COmparing Continuous Optimizers benchmark (COCO). We first analyse the sensitivity of BO to its own parameters, enabling us to answer general questions regarding the choice of the GP kernel or its trend, the initial GP budget, and the suboptimisation of the acquisition function. Then, we study on which function class and dimension BO is relevant when compared to state-of-the-art optimisers for expensive functions.
The second part of this talk describes a new BO algorithm to improve scalability with dimension, called TREGO (trust-region-like efficient global optimisation). TREGO alternates between regular BO steps and local steps within a trust region. By following a classical scheme for the trust region (based on a sufficient decrease condition), we demonstrate that our algorithm enjoys strong global convergence properties, while departing from EGO only for a subset of optimization steps. The COCO benchmark experiments reveal that TREGO consistently outperforms EGO and closes the performance gap with other state-of-the-art algorithms in conditions (high budget and dimension) for which BO was struggling to compete previously.
Joint work Youssef Diouane, Rodolphe Le Riche, Alexandre Scotto Di Perrotolo.
Ref: arXiv:2101.06808 & DiceOptim.
Organizing committee: Julien Bect (L2S), Emmanuel Vazquez (L2S), Didier Clouteau (MSSMAT), Filippo Gatti (MSSMAT), Fernando Lopez Caballero (MSSMAT), Amélie Fau (LMT), Bertrand Iooss (EDF R&D).
Practical details: the seminar will be held online using Microsoft Teams.
If you want to attend this seminar (or any of the forthcoming online UQSay seminars), and if you do not already have access to the UQSay group on Teams, simply send an email and you will be invited. Please specify which email address the invitation must be sent to (this has to be the address associated with your Teams account).
You will find the link to the seminar on the "General" UQSay channel on Teams, approximately 15 minutes before the beginning.
The technical side of things: you can use Teams either directly from you web browser or using the "fat client", which is available for most platforms (Windows, Linux, Mac, Android & iOS). We strongly recommend the latter option whenever possible. Please give it a try before the seminar to anticipate potential problems.
