CCSE Home

Overview

AMReX

People

Publications

Algorithm Development for Black-Box Global Optimization

Optimization

Motivation

Optimization problems arise in many science applications, e.g.

  • Environment: clean up contaminated groundwater at minimal cost
  • Renewable energies: maximize the power generated
  • Simulation: minimize the error between the simulation model and observations

    • Optimization problem characteristics

  • Objective function values are computed by a time consuming black-box simulation model

  • Derivative information is not available

  • Problems are multimodal, i.e., there are several local optima

  • Constraints may be computationally cheap, computed by an expensive black-box simulation model, or simple box constraints

  • The simulation model may not return a function value (not a bug in the simulation model, e.g., may be some exception case in the simulation model)

  • Uncertainty in observation data and stochasticity in forward model evaluation

  • Variables can be continuous, mixed-integer, integer, binary

    • Goal: find a near-optimal solution by doing only very few expensive simulation model evaluations in order to keep the optimization time acceptable

    Challenge

  • Gradient-based methods are not applicable because we do not have derivative information

  • Evolutionary algorithms require too many expensive function evaluations

  • Increasing compute power allows us to increase the accuracy of the simulation models, which leads to increasing compute time for function evaluations

    More sophisticated optimization algorithms are needed to efficiently and effectively tune the simulation model parameters

    • Approach

    We use a computationally cheap approximation s(x) (the surrogate model) of the expensive function f(x) in order to predict function values at unsampled points and guide the search for improved solutions:

    A general surrogate model optimization algorithm works as follows:

  • Step 1: Create an initial experimental design. Evaluate f(x) at the selected points. Fit the surrogate s(x) to the data.

  • Step 2: Solve a computationally cheap auxiliary optimization problem on s(x) to select a new evaluation point xnew and compute f(xnew).

  • Step 3: If the stopping criterion is not satisfied, update s(x) with the new data and go to Step 2. Otherwise, stop.

    • Optimization

    Selected applications

    Combustion simulations: more to come..

    Cosmology: more to come...

    Event generator tuning: more to come...

    Co-optimization of fuels and engines: more to come...

    Ongoing work


  • Development of new algorithms for problems with uncertainty (from noise in observation data, from stochasticity in forward simulation, from model fidelity)

  • Development of algorithms for large-scale problems

  • For more detailed project descriptions, see LBL Optimization

    • Contact

    For more information, contact Juliane Mueller

    Related publications

    T. Takhtaganov, Z. Lukic, J. Mueller, D. Morozov, "Cosmic Inference: Constraining Parameters with Observations and a Highly Limited Number of Simulations", The Astrophysical Journal., 2021. [link].

    J. Mueller, J. Park, R. Sahu, C. Varadharajan, B. Arora, B. Faybishenko, D. Agarwal, "Surrogate optimization of deep neural networks for groundwater predictions", Journal of Global Optimization., 2020. [link].

    J. Mueller, "An algorithmic framework for the optimization of computationally expensive bi-fidelity black-box problems", INFOR: Information Systems and Operational Research., 2019. [link].

    W. Langhans, J. Mueller, and W. Collins, "Optimization of the Eddy-Diffusivity/Mass-Flux shallow cumulus and boundary-layer parameterization using surrogate models", Journal of Advances in Modeling Earth Systems., 2019. [link].

    J. Mueller and M. Day, "Surrogate optimization of computationally expensive black-box problems with hidden constraints", INFORMS Journal on Computing., 2019. [link].

    T. Takhtaganov, J. Mueller "Adaptive Gaussian process surrogates for Bayesian inference", preprint, 2018 [link].

    O. Karslioglu, M. Gehlmann, J. Mueller, S. Nemsak, J. Sethian, A. Kaduwela, H. Bluhm, C.S. Fadley "An Efficient Algorithm for Automatic Structure Optimization in X-ray Standing-Wave Experiments", Journal of Electron Spectroscopy and Related Phenomena, 2018 [link].

    G. Conti, S Nemsak, C.-T. Kuo, M. Gehlmann, C. Conlon, A. Keqi, A. Rattanachata, O. Karslioglu, J. Mueller, J. Sethian, H. Bluhm, J.E. Rault, J.P. Rueff, H. Fang, A. Javey, C.S. Fadley "Characterization of free standing InAs quantum membranes by standing wave hard x-ray photoemission spectroscopy", APL Materials, 2018 [link].

    J. Mueller, J. Woodbury "GOSAC: Global Optimization with Surrogate Approximation of Constraints", Journal of Global Optimization, 2017 [link].

    J. Mueller "SOCEMO: Surrogate Optimization of Computationally Expensive Multi-Objective Problems", INFORMS Journal on Computing, 2017 [link].

    J. Mueller, "MISO: Mixed-Integer Surrogate Optimization Framework", Optimization and Engineering, to appear, 2015 [link].

    J. Mueller, R. Paudel, N. Mahowald, C. Shoemaker, "CH4 Parameter Estimation in CLM4.5bgc Using Surrogate Global Optimization", Geoscientific Model Development, 8:141-207, 2015 [pdf].