Tree Ensembles for Contextual Bandits
Authors: Hannes Nilsson, Rikard Johansson, Niklas Åkerblom, Morteza Haghir Chehreghani
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We further demonstrate the effectiveness of our framework via several experimental studies, employing XGBoost and random forests, two popular tree ensemble methods. Compared to state-of-the-art methods based on decision trees and neural networks, our methods exhibit superior performance in terms of both regret minimization and computational runtime, when applied to benchmark datasets and the real-world application of navigation over road networks. |
| Researcher Affiliation | Collaboration | Hannes Nilsson EMAIL Chalmers University of Technology and University of Gothenburg Rikard Johansson EMAIL Chalmers University of Technology and University of Gothenburg Niklas Åkerblom EMAIL Volvo Car Corporation Chalmers University of Technology and University of Gothenburg Morteza Haghir Chehreghani EMAIL Chalmers University of Technology and University of Gothenburg. The affiliations include academic institutions (Chalmers University of Technology, University of Gothenburg) and an industry corporation (Volvo Car Corporation), indicating a collaboration. |
| Pseudocode | Yes | Algorithm 1 Tree Ensemble Upper Confidence Bound / Tree Ensemble Thompson Sampling |
| Open Source Code | No | The paper does not contain any explicit statement about releasing its own source code, nor does it provide a link to a code repository for the methodology described. |
| Open Datasets | Yes | For this experiment, the data is collected from the UCI machine learning repository (Kelly et al., n.d.), where an overview of the used selected datasets is given in Table 1. Magic (Bock, 2007), which is short for Magic Gamma Telescope, contains only numerical features. The same is true for Shuttle (Statlog (Shuttle), n.d.)... All features of Mushroom (Mushroom, 1987) are categorical, and Adult (Becker & Kohavi, 1996) has a balanced distribution between numerical and categorical features. Edge traversal times have been collected using the Luxembourg SUMO Traffic (Lu ST) simulation scenario (Codeca et al., 2015). |
| Dataset Splits | No | The paper describes how a classification task is transformed into a contextual MAB problem, where contexts are presented sequentially over a certain time horizon (e.g., 10,000 time steps). However, it does not specify explicit training, validation, and test dataset splits in the traditional supervised learning sense for reproducing the data partitioning. |
| Hardware Specification | Yes | The experimental results are obtained using an NVIDIA A40 GPU for Neural UCB and Neural TS, and a desktop CPU for the other agents. |
| Software Dependencies | No | For the implementations, we use the XGBoost library (Chen & Guestrin, 2016) to build gradient-boosted decision trees as our tree ensembles, as well as the version of random forests found in the scikit-learn library (Pedregosa et al., 2011). The paper mentions these software libraries but does not provide specific version numbers for them or any other key software components like Python. |
| Experiment Setup | Yes | For all datasets in Table 1, the neural network agents use a network architecture of one hidden layer with 100 neurons. ...we added a dropout probability of 0.2 when training the neural networks... each of the networks is trained for 10 epochs. For all tree ensemble bandits, we use XGBoost and random forest regressors with MSE loss and ensembles of 100 trees. ...the maximum tree depth, which we set to 10. ...The number of initial random arm selections TI is set to 10 times the number of arms for all tree-based algorithms, and the exploration factor of TEUCB and TETS is set to ν = 1. |