The Kalai-Smorodinsky solution for many-objective Bayesian optimization
Authors: Mickael Binois, Victor Picheny, Patrick Taillandier, Abderrahmane Habbal
JMLR 2020 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our approach is tested on four problems with respectively four, six, eight, and nine objectives. The method is available in the R package GPGame available on CRAN at https://cran.r-project.org/ package=GPGame. Section 4: Experiments This section details numerical experiments on four test problems: two toy problems from the multiobjective and BO literature, a problem of hyperparameter tuning, and the calibration of a complex simulator. |
| Researcher Affiliation | Collaboration | M. Binois EMAIL Mathematics and Computer Science Division, Argonne National Laboratory, Lemont, USA and Université Côte d Azur, Inria, CNRS, LJAD, Sophia Antipolis, France V. Picheny EMAIL PROWLER.io, Cambridge, UK and MIAT, Université de Toulouse, INRA, Castanet-Tolosan, France P. Taillandier EMAIL MIAT, Université de Toulouse, INRA, Castanet-Tolosan, France A. Habbal EMAIL Université Côte d Azur, Inria, CNRS, LJAD, UMR 7351, Parc Valrose, 06108 Nice, France and Mohammed VI Polytechnic University, Benguerir, Morocco. |
| Pseudocode | Yes | Algorithm 1 Pseudo-code for the baseline algorithm for KS Algorithm 2 Pseudo-code for the SUR loop for KS |
| Open Source Code | Yes | Our approach is tested on four problems with respectively four, six, eight, and nine objectives. The method is available in the R package GPGame available on CRAN at https://cran.r-project.org/ package=GPGame. All experiments were conducted in R (R Core Team, 2018), by using the dedicated package GPGame; see the work of Picheny and Binois (2018) for details. |
| Open Datasets | Yes | As a proof of concept, we consider the DTLZ2 function (Deb et al., 2002), with five variables and four objectives, that has a concave dome-shaped Pareto front, and a six variables and six objectives problem obtained by rotating and rescaling six times the classical mono-objective function hartman (Dixon and Szegö, 1978). We consider here the training of a convolutional neural network (CNN) on the classical MNIST data (Le Cun et al., 1998), with 60, 000 handwritten digits for training and an extra 10, 000 for testing. |
| Dataset Splits | Yes | We consider here the training of a convolutional neural network (CNN) on the classical MNIST data (Le Cun et al., 1998), with 60, 000 handwritten digits for training and an extra 10, 000 for testing. A validation data set is extracted from the training data to monitor overfitting, representing 20% of the initial size of the training data. |
| Hardware Specification | Yes | repeating five times each experiment, which takes up to 30 minutes on a desktop with a 3.2 Ghz quad-core processor and 4 Gb of RAM. approximately 30 minutes per run on a desktop computer with a 3.60 GHz eight-core processor and 32 Go RAM |
| Software Dependencies | Yes | All experiments were conducted in R (R Core Team, 2018), by using the dedicated package GPGame; see the work of Picheny and Binois (2018) for details. We use the keras package (Allaire and Chollet, 2018) to interface with the high-level neural networks API Keras (Chollet et al., 2015) to create and train a CNN. |
| Experiment Setup | Yes | The six hyperparameters to tune, detailed in Table 2 in Appendix C, along with their range of variation, are the number of filters and dropout rates of each layer, plus the number of units of the last hidden layer and number of epochs. GPs are trained by using Matérn 5/2 kernels with an estimated linear trend. An initial 100-point optimized LHD is generated, which is used to fit GPs (constant trend, Matérn 5/2 anisotropic covariance). |