Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Trading Convergence Rate with Computational Budget in High Dimensional Bayesian Optimization
Authors: Hung Tran-The, Sunil Gupta, Santu Rana, Svetha Venkatesh2425-2432
AAAI 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform empirical experiments to evaluate our method extensively, showing that its sample efficiency is better than the existing methods for many optimisation problems involving dimensions up to 5000. |
| Researcher Affiliation | Academia | Hung Tran-The, Sunil Gupta, Santu Rana, Svetha Venkatesh Applied Artificial Intelligence Institute,Deakin University, Geelong, Australia EMAIL |
| Pseudocode | Yes | Algorithm 1 MS-UCB Algorithm |
| Open Source Code | No | The paper does not provide any explicit statement or link regarding the open-sourcing of the code for the methodology described. |
| Open Datasets | Yes | We use the Gisette dataset from the UCI repository (Newman and Merz 1998) with dimension D = 5000. |
| Dataset Splits | No | The paper mentions 'validation loss' in the context of neural network parameter search, implying a validation set was used, but it does not provide specific details on the size, percentage, or methodology of the train/validation/test dataset splits for reproducibility. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory, or cloud instance types) used for running its experiments. |
| Software Dependencies | No | The paper states 'We implemented our proposed MS-UCB, Line BO, Dropout UCB and SRE in Python 3 using GPy,' but it does not specify exact version numbers for Python or GPy, which is required for reproducible software dependencies. |
| Experiment Setup | Yes | Each algorithm was randomly initialized with 20 points. To maximise the acquisition function, we used LBFGS-B algorithm with 10 D random starts. For Gaussian process, we used Matern kernel and estimated the kernel hyper-parameters automatically from data. We choose d = 5 for all methods except Line BO for which d = 1 is a requirement and the GP-UCB which works directly in original D-dimensional space. We use N0 = 1, α = 0 as parameters for our method. |