Stochastic Variance-Reduced Cubic Regularization Methods
Authors: Dongruo Zhou, Pan Xu, Quanquan Gu
JMLR 2019 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical experiments with different nonconvex optimization problems conducted on real datasets validate our theoretical results for both SVRC and Lite-SVRC. |
| Researcher Affiliation | Academia | Dongruo Zhou EMAIL Department of Computer Science University of California, Los Angeles Los Angeles, CA 90095, USA Pan Xu EMAIL Department of Computer Science University of California, Los Angeles Los Angeles, CA 90095, USA Quanquan Gu EMAIL Department of Computer Science University of California, Los Angeles Los Angeles, CA 90095, USA |
| Pseudocode | Yes | Algorithm 1 Stochastic Variance Reduction Cubic Regularization (SVRC) ... Algorithm 2 Sample efficient stochastic variance-reduced cubic regularization method (Lite-SVRC) |
| Open Source Code | No | The paper does not contain any explicit statement about providing source code for the methodology described in this paper, nor does it provide a link to a code repository. It discusses other algorithms' code in comparison sections but not its own. |
| Open Datasets | Yes | We investigate two nonconvex problems on three different datasets, a9a, ijcnn1 and covtype, which are all common datasets used in machine learning and the sizes are summarized in Table 3. |
| Dataset Splits | No | The paper mentions 'Given training data xi Rd and label yi {0, 1}, 1 i n' in the context of problem formulation but does not provide details on how the datasets (a9a, ijcnn1, covtype) were split into training, validation, or test sets for the experiments. |
| Hardware Specification | No | The paper mentions 'CPU time' in the experimental results but does not specify any hardware details such as GPU models, CPU types, or memory used for running the experiments. |
| Software Dependencies | No | The paper mentions using a 'Lanczos-type method' for solving the cubic sub-problem, as used by Kohler and Lucchi (2017), but does not specify any software libraries or packages with version numbers used for its implementation. |
| Experiment Setup | Yes | Parameters: For each algorithm and each dataset, we choose different bg, bh, T for the best performance. Meanwhile, we choose the cubic regularization parameter as Ms,t = α/(1+β)(s+t/T), α, β > 0 for each iteration. ... For the binary logistic regression problem in (26), the parameters of Ms,t = α/(1 + β)(s+t/T), α, β > 0 are set as follows: α = 0.05, β = 0 for a9a and ijcnn1 datasets and α = 5e3, β = 0.15 for covtype. ... For the non-linear least squares problem in (27), we set α = 0.05, 1e8, 0.003 and β = 0, 1, 0.5 for a9a, covtype and ijcnn1 datasets respectively. ... we set λ = 10 3 for all three datasets, and set γ = 10, 50, 100 for a9a, ijcnn1 and covtype datasets respectively. ... λ = 5 10 3 for all three datasets, and γ = 10, 20, 50 for a9a, ijcnn1 and covtype datasets respectively. |