Asynchronous Online Testing of Multiple Hypotheses
Authors: Tijana Zrnic, Aaditya Ramdas, Michael I. Jordan
JMLR 2021 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We illustrate our algorithms in simulations by comparing to existing algorithms for online FDR control. Here we present the results of several numerical simulations, which show the gradual change in performance of LORD* and SAFFRON* with the increase of asynchrony and the lags of local dependence. |
| Researcher Affiliation | Academia | Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA 94720, USA. Department of Statistics and Data Science Carnegie Mellon University Pittsburgh, PA 15232, USA. |
| Pseudocode | Yes | Algorithm 1 The LORD++ algorithm under general conflict sets (a special case of LORD*) |
| Open Source Code | Yes | 3. The code for all experiments in this section is available at: https://github.com/tijana-zrnic/async-online-FDR-code |
| Open Datasets | Yes | We perform an additional case study on a high-throughput phenotypic data set from the International Mouse Phenotyping Consortium (IMPC) data repository. We use the subset of the database organized by Robertson et al. (2019), which is available at https://zenodo.org/record |
| Dataset Splits | No | The paper uses synthetic data generated based on Gaussian observations and a real dataset from the International Mouse Phenotyping Consortium (IMPC) which is split into batches for analysis, but does not provide specific train/test/validation splits for model training and evaluation. |
| Hardware Specification | No | The paper describes numerical simulations and experiments in Section 7 and Appendix D, but does not specify any particular hardware used for these computations (e.g., CPU, GPU models, or cloud resources). |
| Software Dependencies | No | The paper provides a link to source code on GitHub but does not explicitly list specific software dependencies with version numbers within the text. |
| Experiment Setup | Yes | In all of the simulations we present the FDR is controlled at α = 0.05, and we estimate the FDR and power by averaging the results of 200 independent trials. The SAFFRON-type algorithms use the constant candidacy threshold sequence λ = 1/2, across all tests. The LORD-type algorithms use the LORD++ update for test levels. |