Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Neyman-Pearson Classification under High-Dimensional Settings
Authors: Anqi Zhao, Yang Feng, Lie Wang, Xin Tong
JMLR 2016 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Besides their desirable theoretical properties, we also demonstrated their numerical advantages in prioritized error control via both simulation and real data studies. |
| Researcher Affiliation | Academia | Anqi Zhao EMAIL Department of Statistics Harvard University Yang Feng EMAIL Department of Statistics Columbia University Lie Wang EMAIL Department of Mathematics Massachusetts Institute of Technology Xin Tong EMAIL Department of Data Sciences and Operations Marshall Business School University of Southern California |
| Pseudocode | Yes | Definition 2.4. Neyman-Pearson plug-in procedure Step 1 Use S1 1, S1 2, S0 1, and S0 2 to construct a density ratio estimate ˆr. The specific use of each subsample will be introduced in Section 2.4. Step 2 Given ˆr, choose a threshold estimate b Cα from the set ˆr(S0 3) = {ˆr(Vi+m1+m2)}m3 i=1. |
| Open Source Code | No | The paper mentions using "R package e1071 for nb and svm, and the R package glmnet for pen-log" which are third-party tools, but does not provide any statement or link for the authors' own implementation code for the described methodology. |
| Open Datasets | Yes | The neuroblastoma dataset introduced by Oberthuer et al. (2006) provides a perfect illustration of such intuition. The p53 mutants dataset (Danziger et al., 2006) contains d = 5407 attributes extracted from biophysical experiments for 16772 mutant p53 proteins... Now, we consider an e-mail spam dataset available at https://archive.ics.uci.edu/ml/ datasets/Spambase, which contains 4601 observations with 57 features... |
| Dataset Splits | Yes | The neuroblastoma dataset...We randomly selected 41 0 s and 123 1 s as our training sample (such that the proportion of 0 s is about the same as that in the entire dataset), and tested the resulting classifiers on the rest 15 0 s and 67 1 s. p53 mutants dataset...This dataset is split into a training set with 100 observations from the active class and 1000 observations from the inactive class, and a testing set with the remaining observations. Email spam dataset...This augmented dataset is split into a training set with 1000 observations from each class and a testing set with the remaining observations. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory specifications) used for running the experiments. It only describes simulation settings and data analysis. |
| Software Dependencies | No | The paper mentions using "R package e1071 for nb and svm, and the R package glmnet for pen-log." However, it does not specify version numbers for these R packages, which are crucial for reproducibility. |
| Experiment Setup | Yes | To train the classifiers in Table 2, we set α = 0.05, δ1 = 0.05, and δ3 = 0.05 throughout this section unless specified otherwise. In Assumption 1, motivated by Proposition 2.5, we take m1 = min{10 log(4d/δ1), m/4}1I(screening), n1 = min{10 log(4d/δ1), n/2}1I(screening), m2 = m/2 m1, n2 = n n1, and m3 = m m/2 . Q is set at 0.95 throughout this section. |