Hierarchical Kernels in Deep Kernel Learning
Authors: Wentao Huang, Houbao Lu, Haizhang Zhang
JMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To verify the theoretical results in the paper, we shall conduct initial experiments with hierarchical Gaussian kernels and hierarchical exponential kernels in this section. Specifically, we shall evaluate the hierarchical Gaussian kernels from consecutive composition with the exponential function on various tasks, including classification on the Scikit-learn (Pedregosa et al., 2011) moon scattering dataset, classification on CIFAR-10, and regression on UCI datasets. Considering the numerical stability, we shall only evaluate hierarchical kernels with at most three layers, and shall slightly modify the hierarchical Gaussian kernels as Gk+1(x, y) = exp (ek(1) (Gk(x, y) 1)) , G0(x, y) = exp( λ x y 2), x, y Rd. (59) To further confirm the theoretical results of the paper, we shall also evaluate the ℓ1 norm hierarchical exponential kernels on three regression tasks from LIBSVM (Chang and Lin, 2011). |
| Researcher Affiliation | Academia | Wentao Huang EMAIL Houbao Lu EMAIL Haizhang Zhang EMAIL School of Mathematics (Zhuhai) Sun Yat-sen University Zhuhai 519082, P. R. China |
| Pseudocode | No | The paper does not contain any sections explicitly labeled as "Pseudocode" or "Algorithm", nor does it present any structured, code-like procedural steps. |
| Open Source Code | Yes | our codes could be accessed via the github repository https://github.com/Saeba Huang/ Hierarchical-Kernel-in-Deep-Kernel-Learning. |
| Open Datasets | Yes | Specifically, we shall evaluate the hierarchical Gaussian kernels from consecutive composition with the exponential function on various tasks, including classification on the Scikit-learn (Pedregosa et al., 2011) moon scattering dataset, classification on CIFAR-10, and regression on UCI datasets. [...] To further confirm the theoretical results of the paper, we shall also evaluate the ℓ1 norm hierarchical exponential kernels on three regression tasks from LIBSVM (Chang and Lin, 2011). |
| Dataset Splits | Yes | We randomly choose 500 points as the training set which is equally divided into two classes. [...] For each evaluation, we normalize the data to be between 1 and 1, and perform 5-fold cross-validation. |
| Hardware Specification | No | The paper mentions using "thundersvm" which is described in a citation as running "on GPUs and CPUs", but it does not specify the particular GPU or CPU models, memory, or other hardware details used for the experiments conducted in this paper. |
| Software Dependencies | No | The paper mentions using "thundersvm" (Wen et al., 2018) and the "SVM module in Scikit-learn" but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | For classification tasks, we shall use the C-support vector classification (Boser et al., 1992; Cortes and Vapnik, 1995) method [...] For regression tasks, we shall use the ϵ-support vector regression method [...] As our focus is on the hierarchical kernels, we shall fix C = 1 and ε = 10 3 in all the tasks for fair comparison. For each task and each hierarchical kernel, the hyperparameter λ in the hierarchical kernels (59) and (60) will be optimally chosen. For evaluations with hierarchical Gaussian kernels, we implement with the thundersvm (Wen et al., 2018) to accelerate the training process. [...] For hierarchical Guassian kernel Gk, 0 k 3, the hyperparameter λ will be optimally chosen from [2 5, 2 4, , 29, 210]. For evaluations on CIFAR-10 dataset with hierarchical Gaussian kernels, the λ of each layer is optimally chosen from [2 19, , 2 18, , 2 8]. |