Robust Frequent Directions with Application in Online Learning
Authors: Luo Luo, Cheng Chen, Zhihua Zhang, Wu-Jun Li, Tong Zhang
JMLR 2019 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental studies demonstrate that the proposed method outperforms state-of-the-art second order online learning algorithms. Keywords: Matrix approximation, sketching, frequent directions, online convex optimization, online Newton algorithm |
| Researcher Affiliation | Academia | Luo Luo EMAIL Cheng Chen JACK EMAIL Department of Computer Science and Engineering Shanghai Jiao Tong University 800 Dongchuan Road, Shanghai, China 200240 Zhihua Zhang EMAIL National Engineering Lab for Big Data Analysis and Applications School of Mathematical Sciences Peking University 5 Yiheyuan Road, Beijing, China 100871 Wu-Jun Li EMAIL National Key Laboratory for Novel Software Technology Collaborative Innovation Center of Novel Software Technology and Industrialization Department of Computer Science and Technology Nanjing University 163 Xianlin Avenue, Nanjing, China 210023 Tong Zhang EMAIL Computer Science & Mathematics Hong Kong University of Science and Technology Hong Kong |
| Pseudocode | Yes | Algorithm 1 Frequent Directions; Algorithm 2 Robust Frequent Directions; Algorithm 3 RFD for Online Newton Step; Algorithm 4 Fast Frequent Directions; Algorithm 5 Fast Robust Frequent Directions; Algorithm 6 Fast RFD for Online Newton Step; Algorithm 8 Greedy Low-rank Approximation |
| Open Source Code | No | The paper mentions that "The experiments are conducted in Matlab" but does not provide any statement about releasing the source code for their proposed methods or a link to a code repository. |
| Open Datasets | Yes | In this section, we evaluate the performance of robust frequent directions (RFD) and online Newton step by RFD (RFD-SON) on six real-world data sets a9a, gisette, sido0, farm-ads, rcv1 and real-sim, whose details are listed in Table 1. The data sets sido0 and farm-ads can be found on Causality Workbench3, and UCI Machine Learning Repository4. The others can be downloaded from LIBSVM repository5. |
| Dataset Splits | Yes | We use 70% of the data set for training and the rest for test. |
| Hardware Specification | Yes | The experiments are conducted in Matlab and run on a server with Intel (R) Core (TM) i7-3770 CPU 3.40GHz 2, 8GB RAM and 64-bit Windows Server 2012 system. |
| Software Dependencies | No | The paper states, "The experiments are conducted in Matlab" but does not specify a version number for Matlab or any other software dependencies with version numbers. |
| Experiment Setup | Yes | The hyperparameter α0 is tuned from {10 3, 10 2 . . . 105, 106} for all methods and let η = 1/t for SON algorithms. For PFD-SON, we let β = 0.2 heuristically because it usually achieves good performance on approximating the covariance matrix. Additionally, RFD-SON includes the result with α0 = 0 (RFD0-SON). The sketch size of sketched online Newton methods is chosen from {5, 10, 20} for a9a, gisette, sido0, and {20, 30, 50} for farm-ads, rcv1 and real-sim. |