A Tight Bound of Hard Thresholding
Authors: Jie Shen, Ping Li
JMLR 2017 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we present a comprehensive empirical study for the proposed HT-SVRG algorithm on two tasks: sparse recovery (compressed sensing) and image classification. The experiments on sparse recovery is dedicated to verifying the theoretical results we presented, and we visualize the classification models learned by HT-SVRG to demonstrate the practical efficacy. |
| Researcher Affiliation | Collaboration | Jie Shen EMAIL Department of Computer Science Rutgers University Piscataway, NJ 08854, USA Ping Li EMAIL Baidu Research Bellevue, WA 98004, USA |
| Pseudocode | Yes | Algorithm 1 Hard Thresholded Stochastic Variance Reduced Gradient Method (HT-SVRG) |
| Open Source Code | No | No explicit statement or link to source code for the methodology described in this paper is provided. The paper only mentions a concurrent work by Li et al. (2016) that made their work public. |
| Open Datasets | Yes | Here, we study the performance on a realistic image dataset MNIST1, consisting of 60 thousands training samples and 10 thousands samples for testing. There is one digit on each image of size 28-by-28, hence totally 10 classes. Some of the images are shown in Figure 6. The update frequency m is fixed as m = 3n. We compute the heuristic step size η as in the previous section, i.e., η = 2/svds(AA ) 10 3. Since for the real-world dataset, the true sparsity is actually unknown, we tune the sparsity parameter k and study the performance of the algorithm. 1. http://yann.lecun.com/exdb/mnist/ |
| Dataset Splits | Yes | Here, we study the performance on a realistic image dataset MNIST1, consisting of 60 thousands training samples and 10 thousands samples for testing. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory amounts, or detailed computer specifications) used for running its experiments are provided in the paper. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., library or solver names with version numbers) are mentioned in the paper. |
| Experiment Setup | Yes | If not specified, we use m = 3n, k = 9K, and S = 10000 for HT-SVRG. We also use the heuristic step size η = 2/svds(AA ) for HT-SVRG and PGD, where svds(AA ) returns the largest singular value of the matrix AA . For the MNIST dataset: "The update frequency m is fixed as m = 3n. We compute the heuristic step size η as in the previous section, i.e., η = 2/svds(AA ) 10 3." |