Set-valued Classification with Out-of-distribution Detection for Many Classes
Authors: Zhou Wang, Xingye Qiao
JMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Both theoretical analysis and numerical experiments are conducted to illustrate the effectiveness of the proposed method. |
| Researcher Affiliation | Academia | Zhou Wang EMAIL Department of Mathematics and Statistics Binghamton University, State University of New York Binghamton, NY 13902, USA; Xingye Qiao EMAIL Department of Mathematics and Statistics Binghamton University, State University of New York Binghamton, NY 13902, USA |
| Pseudocode | Yes | Appendix B.1 Outline of the Kernel Feature Selection Algorithm. Algorithm 1 Weighted Kernel Feature Selection |
| Open Source Code | Yes | The code is publicly available at https://github.com/Zhou198/GPS. |
| Open Datasets | Yes | In this section, we conduct comparisons by considering three real data sets: Zipcode, Phoneme, and Cifar10. |
| Dataset Splits | Yes | Specifically, let the decision rule ˆfk learned from each method be the conformal score function. Given any class k, we randomly split the data into the estimation set and the calibration set. We use the first set to train the classifier and the score function, and the second part to conduct the calibration and parameter tuning. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments with specific models or processors. |
| Software Dependencies | No | For the GPS methods, we use cvxopt and scipy in Python to solve the convex optimization problems involved. For competing set-valued classification methods... we use their implementations in the scikit-learn library. We use the Py Torch library pytorch-ood... The paper mentions software tools like Python, cvxopt, scipy, scikit-learn, PyTorch, and pytorch-ood, but does not provide specific version numbers for any of them. |
| Experiment Setup | Yes | To choose the tuning parameters, the candidate hyper-parameters C1, C2 in GPSKFS is searched from grid {1, 2, 3} and 10 { -2, -1.5, -1, -0.5, 0}, respectively. The hyperparameter C in GPS is searched from the grid 10 { -2, -1.5, -1, -0.5, 0}. For the σ parameter in the Gaussian kernel exp( d x d x 2/σ2), we choose it from the {25, 37.5, 50, 62.5, 75}-th percentiles of all the pairwise Euclidean distances between the weighted training sample d (x x ) 2, where d is 1 in GPS, or is the currently estimated weight vector which can itself evolve in the iterations in GPSKFS. For BCOPS-RF, the maximum depth of the tree is searched from {10, 20, . . . , 90, 100}. Minimum samples to split an internal node, minimum samples at a leaf node, and the number of trees are searched from {2, 5, 10}, {2, 4, 6}, and {50, 150, 200}, respectively. |