Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Minimax Optimal Fixed-Budget Best Arm Identification in Linear Bandits
Authors: Junwen Yang, Vincent Tan
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, numerical experiments demonstrate considerable empirical improvements over existing algorithms on a variety of real and synthetic datasets. |
| Researcher Affiliation | Academia | Junwen Yang Institute of Operations Research and Analytics National University of Singapore EMAIL Vincent Y. F. Tan Department of Mathematics Department of Electrical and Computer Engineering Institute of Operations Research and Analytics National University of Singapore EMAIL |
| Pseudocode | Yes | Algorithm 1 Optimal Design-based Linear Best Arm Identification (OD-Lin BAI) |
| Open Source Code | Yes | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] See Appendix E and the code in the supplementary. |
| Open Datasets | Yes | We present the results of one synthetic dataset here. Additional implementation details and numerical results (including another synthetic dataset, one real-world dataset and comparison to the recent LT&S algorithm for best arm identification in linear bandits with fixed confidence [33]) are provided in Appendix E. ... This benchmark dataset, in which there are numerous competitors for the second best arm, was considered for the problem of best arm identification in linear bandits in the fixed-confidence setting [30, 31, 33]. |
| Dataset Splits | No | The paper describes generating synthetic datasets and averaging results over independent trials, but does not specify traditional training, validation, or test dataset splits. |
| Hardware Specification | No | The paper states in its checklist that hardware specifications are included, but no specific hardware details (such as GPU or CPU models) are provided within the main text of the paper. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | The experimental results with fixed T and K are presented in Figure 1 and Figure 2 respectively. In each setting, the reported error probabilities of different algorithms are averaged over 1024 independent trials... We assume that the additive random noise follows the standard Gaussian distribution N(0, 1). For simplicity, we set the unknown parameter vector θ = [1, 0]. |