A permutation-free kernel two-sample test
Authors: Shubhanshu Shekhar, Ilmun Kim, Aaditya Ramdas
NeurIPS 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We now present experimental validation of the theoretical claims of the previous section. In particular, our experiments demonstrate that (i) the limiting null distribution of x\ MMD 2 is N(0, 1) under a wide range of choices of dimension d, sample sizes n, m and the kernel k, and (ii) the power of our x MMD test is competitive with the kernel-MMD permutation test. We now describe the experiments in more detail. Additional experimental results are reported in Appendix E. |
| Researcher Affiliation | Academia | Shubhanshu Shekhar Department of Statistics and Data Science Carnegie Mellon University Pittsburgh, PA 15213 EMAIL Ilmun Kim Department of Statistics and Data Science Department of Applied Statistics Yonsei University EMAIL Aaditya Ramdas Department of Statistics and Data Science Machine Learning Department Carnegie Mellon University EMAIL |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | We will include the code with the supplemental material. |
| Open Datasets | Yes | The statistic is computed with Gaussian kernel (ksn(x, y) = exp( sn x y 2 2)) with scale parameter sn chosen by the median heuristic for different choices of n, m and d, and with samples X and Y drawn from a multivariate Gaussian distribution with identity covariance matrix. We consider the two-sample testing problem with P = N(0, Id) Q = N(aϵ,j, Id) for different choices of ϵ and d and j. |
| Dataset Splits | No | The paper describes generating synthetic data from specified distributions for experiments rather than using predefined train/validation/test splits of a fixed dataset. |
| Hardware Specification | Yes | The experiments were run on a workstation, whose specifications are provided in Appendix E |
| Software Dependencies | No | The paper does not provide specific software names with version numbers needed to replicate the experiment. |
| Experiment Setup | Yes | The statistic is computed with Gaussian kernel (ksn(x, y) = exp( sn x y 2 2)) with scale parameter sn chosen by the median heuristic for different choices of n, m and d, and with samples X and Y drawn from a multivariate Gaussian distribution with identity covariance matrix. We compare the performance of our proposed test Ψ with the kernel-MMD permutation test, implemented with B = 200 permutations. |