Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees
Authors: Jonathan Brophy, Zayd Hammoudeh, Daniel Lowd
JMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on 22 real-world data sets to compare eight different influence-estimation methods using 5 different evaluation measures and 4 popular modern GBDT implementations. |
| Researcher Affiliation | Academia | Jonathan Brophy EMAIL Zayd Hammoudeh EMAIL Daniel Lowd EMAIL Department of Computer and Information Science University of Oregon Eugene, OR 97403, USA |
| Pseudocode | No | The paper describes methods with mathematical formulations and prose, but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | source code is available at https://github.com/jjbrophy47/tree_influence. |
| Open Datasets | Yes | We evaluate on 22 real-world tabular data sets (13 binary-classification tasks, 1 multiclass-classification task, and 8 regression tasks) well-suited for boosted tree-based models... All additional data set details are in the Appendix, A.1. ... Adult (Dua and Graff, 2019) contains 48,842 instances... Bank (Moro et al., 2014; Dua and Graff, 2019) consists of 41,188 marketing phone calls... |
| Dataset Splits | Yes | For any data set without a predefined train/test split, we sample 80% of the data uniformly at random for training and use the rest for testing... Each model is tuned using 5-fold cross-validation |
| Hardware Specification | Yes | Experiments are run on an Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.6GHz with 100GB of DDR4 RAM @ 2.4GHZ. |
| Software Dependencies | Yes | Experiments are run using Python 3.9.6, and source code for all influence-estimation implementations and all experiments is available at https://github.com/jjbrophy47/tree_influence. |
| Experiment Setup | Yes | Each model is tuned using 5-fold cross-validation; selected hyperparameters are in A.2: Table 4, with predictive performance comparisons in A.2: Table 3. ... For the LGB, XGB, CB, and SGB models, we tune the number of trees/boosting iterations (T) using values [10, 25, 50, 100, 200]. Since the LGB and SGB models grow trees in a leaf-wise (depth-first) manner, we tune the maximum number of leaves (lmax) for LGB and SGB using values [15, 31, 61, 91]. In contrast, we tune the the maximum depth (dmax) for XGB and CB using values [2, 3, 4, 5, 6, 7]. We also tune the learning rate (η) for CB using values [0.1, 0.3, 0.6, 0.9], and the maximum number of bins (bmax) for SGB using values [50, 100, 250]. We tune all hyperparameters using 5-fold cross-validation. |