Utilitarian Algorithm Configuration for Infinite Parameter Spaces
Authors: Devon Graham, Kevin Leyton-Brown
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We discuss empirical evaluations in Section 5 and conclude with Section 6. We perform our experiments on three existing datasets from the algorithm configuration literature. The minisat dataset represents the measured execution times of the SAT solver by that name on a generated set of instances (see Weisz et al. (2018) for details). |
| Researcher Affiliation | Academia | Devon R. Graham & Kevin Leyton-Brown Department of Computer Science University of British Columbia Vancouver, BC EMAIL |
| Pseudocode | Yes | Algorithm 1 OUP 1: Input: configurations i = 1, ..., n; instances j = 1, 2, ...; utility function u; failure probability δ. 2: I {1, ..., n} . candidate configurations ... Algorithm 2 COUP 1: Input: distribution over configurations DA; instances j = 1, 2, ...; utility function u; failure parameter δ; phase parameters { p, γp}p=1,2,3,.... |
| Open Source Code | Yes | Code to reproduce all experiments can be found at https://github.com/drgrhm/coup. |
| Open Datasets | Yes | We perform our experiments on three existing datasets from the algorithm configuration literature. The minisat dataset represents the measured execution times of the SAT solver by that name on a generated set of instances (see Weisz et al. (2018) for details). The cplex rcw and cplex region datasets represent the predicted solve times of the CPLEX integer program solver on wildlife conservation and combinatorial auction problems, respectively (see Weisz et al. (2020) for details). |
| Dataset Splits | No | The paper mentions using specific datasets (minisat, cplex rcw, cplex region) and instances within them, but does not specify any training/test/validation splits or their proportions for the experiments conducted in the paper. The focus is on evaluating algorithm configurations across a set of inputs of interest, not on splitting a dataset for model training/evaluation. |
| Hardware Specification | No | This work was funded by an NSERC Discovery Grant, a CIFAR Canada AI Research Chair (Alberta Machine Intelligence Institute), and computational resources provided both by UBC Advanced Research Computing and a Digital Research Alliance of Canada RAC Allocation. No specific hardware details (e.g., CPU, GPU models) are provided. |
| Software Dependencies | No | The paper mentions the 'CPLEX integer program solver' in the context of datasets, but does not provide specific software dependencies with version numbers for the authors' own implementation or experimental setup. |
| Experiment Setup | Yes | The log-Laplace utility function is defined as u LL(t) = 1 − 1 $ if t < 0 and u LL(t) = 1 $ otherwise. The uniform utility function is defined as uunif(t) = 1 t 0 if t < 0 and uunif(t) = 0 otherwise. The parameter 0 is set to 60 in both cases, and the parameter is set to 1. To facilitate the comparison with UP we use the original doubling condition when recreating our previous experiments (Graham et al., 2023b, Figures 1 and 2). For other experiments we use the improved doubling condition. We set δ = 0.01 throughout. For each phase p we set p = e−p/6 and γp = e−p/3. |