PABBO: Preferential Amortized Black-Box Optimization
Authors: Xinyu Zhang, Daolang Huang, Samuel Kaski, Julien Martinelli
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On a benchmark composed of synthetic and real-world datasets, our method is several orders of magnitude faster than the usual Gaussian process-based strategies and often outperforms them in accuracy. ... We evaluate PABBO on synthetic functions, described in Section 5.1, and against real-world examples from the BO literature: hyperparameter optimization (Section 5.2) and human preferences datasets (Section 5.3). Subsequently, Section 5.4 presents a series of ablation studies... |
| Researcher Affiliation | Academia | Xinyu Zhang , Daolang Huang , Samuel Kaski Department of Computer Science Aalto University EMAIL Julien Martinelli Inserm Bordeaux Population Health Vaccine Research Institute Universit e de Bordeaux Inria Bordeaux Sud-ouest |
| Pseudocode | Yes | Algorithm 1 Preferential Amortized Black-box Optimization (PABBO) ... Algorithm S1 PABBO test time algorithm |
| Open Source Code | Yes | The code to reproduce our experiments is available at https://github.com/ xinyuzc/PABBO. |
| Open Datasets | Yes | The HPO-B benchmark contains multiple search spaces... (Pineda Arango et al., 2021)... The Sushi dataset collects a preference score... Available at https://www.kamishima.net/sushi/ |
| Dataset Splits | Yes | All the meta-datasets within a search space have been categorized into three splits: meta-train, meta-validation, and meta-test. ... Each meta-dataset is equally divided beforehand into two parts from which we sample either prediction set D(p) or query set D(q), so as to prevent any information leak from rewards. |
| Hardware Specification | Yes | We train our model using up to 5 Tesla V100-SXM2-32GB GPUs... We evaluate all the models on 2x64 core AMD EPYC 7713 @2.0 GHz. |
| Software Dependencies | No | PABBO is implemented using Py Torch (Paszke et al., 2019). ... For q EUBO, q EI, q NEI, we used the implementation from the Bo Torch library (Balandat et al., 2020). |
| Experiment Setup | Yes | Hyperparameter settings can be found in Appendix D.1. For q EUBO, q EI, q NEI, we used the implementation from the Bo Torch library (Balandat et al., 2020). |