Bayesian optimization is a popular optimization method for expensive functions.

Key References:

• Calandra, R.; Seyfarth, A.; Peters, J. & Deisenroth, M. P.
  Bayesian Optimization for Learning Gaits under Uncertainty
  Annals of Mathematics and Artificial Intelligence (AMAI), 2015, 76, 5-23
• Letham, B.; Calandra, R.; Rai, A. & Bakshy, E.
  Re-Examining Linear Embeddings for High-dimensional Bayesian Optimization
  Advances in Neural Information Processing Systems (NeurIPS), 2020