Pairwise Learning with Adaptive Online Gradient Descent
Authors: Tao Sun, Qingsong Wang, Yunwen Lei, Dongsheng Li, Bao Wang
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we numerically validate our theoretical findings for both convex and nonconvex cases. To this end, we compare our proposed AOGD against the baseline OGD proposed in (Yang et al., 2021b) 6 for pairwise learning in terms of generalization and rate of convergence with respect to the number of iteration. We also include the results of Ada OAM in (Ding et al., 2015), an adaptive online algorithm for AUC maximization. |
| Researcher Affiliation | Academia | College of Computer National University of Defense Technology Changsha, Hunan, China; Department of Mathematics Scientific Computing and Imaging (SCI) Institute University of Utah Salt Lake City, Utah, USA; Department of Mathematics The University of Hong Kong Pokfulam, Hong Kong |
| Pseudocode | Yes | Algorithm 1 Online Gradient Descent (OGD) for Pairwise Learning (Yang et al., 2021b); Algorithm 2 Adaptive Online Gradient Descent (AOGD) for Pairwise Learning |
| Open Source Code | No | The paper references an existing GitHub repository for OGD (https://github.com/zhenhuan-yang/simple-pairwise) on which their implementation is based, but it does not explicitly state that the source code for the proposed AOGD method itself is open-source or provided. |
| Open Datasets | Yes | Table 1: Statistics of the dataset used for contrasting the performance of AOGD and OGD, where n is the number of samples in each dataset, and d is the number of features of each instance in a given dataset. All datasets come from the LIBSVM website (Chang & Lin, 2011), and they are used in (Yang et al., 2021b). |
| Dataset Splits | Yes | For each dataset, we use 80% of the data for training and the remaining 20% for testing. All the reported results are based on 25 runs with random shuffling. To determine proper hyperparameters for OGD, AOGD, and Ada OAM, we conduct 5-fold cross-validation on the training sets. |
| Hardware Specification | No | No specific hardware details (like GPU models, CPU types, or memory amounts) used for running the experiments are mentioned in the paper. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers for its implementation. While machine learning frameworks like Python and PyTorch are commonly used, no versions are specified. |
| Experiment Setup | Yes | To determine proper hyperparameters for OGD, AOGD, and Ada OAM, we conduct 5-fold cross-validation on the training sets: 1) for OGD, we select stepsizes ηt = η 10[ 5:5]7 and the parameter space K is set to be the L2-ball centered at the origin with radius R 10[ 3,3]; 2) for AOGD, we let θ = 0.9 and we select stepsizes ηt = η 10[ 5:5] and the parameter space K is also the L2-ball centered at the origin with radius R 10[ 3,3]; 3) for Ada OAM, we select stepsizes ηt = η 10[ 5:5] and the parameter space K is set to be the L2-ball centered at the origin with radius R 10[ 3,3]. |