Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Optimal Rates of Distributed Regression with Imperfect Kernels
Authors: Hongwei Sun, Qiang Wu
JMLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we study the distributed kernel regression via the divide and conquer approach... We focus on the use of kernel ridge regression (KRR) and bias corrected kernel ridge regression (BCKRR) in the divide and conquer approach... We generate N = 4098 sample points and use number of partitions m {2, 4, 8, 16, 32, 64, 128, 256, 512, 1024}. Mean squared errors between the estimated function and the true regression function is used to measure the performance. According to Zhang et al. (2015), if the Soboleve space kernel KS is used, the theoretically optimal choice of the regularization parameter is λ = N 2 3 . When Guassian kernel is used, by Theorem 1 and Theorem 2, the optimal choice of λ should depend on the index r in the source condition, which, unfortunately, is unknown. By 0 < r < 1 2 we know the optimal choice should be N α with 1 2 < α < 1 and α = 2 3 seems an acceptable choice. So we will also use λ = N 2 3 for the Guassian kernel to make the first comparison between the four distributed kernel regression algorithms... Figure 1: Mean squared error of distributed kernel regression with Sobolev space kernel and Guassian kernel when (a) the regularization parameters is fixed as λ = N 2 3 and (b) the regularization parameter is locally tuned and underregularized according to equation (8). |
| Researcher Affiliation | Academia | Hongwei Sun EMAIL School of Mathematical Sciences, University of Jinan Jinan, Shandong, P. R. China Qiang Wu EMAIL Department of Mathematical Sciences, Middle Tennessee State University Murfreesboro, TN 37132, USA |
| Pseudocode | No | The paper describes algorithms (KRR, BCKRR) and their processes in prose, but it does not present them in a structured pseudocode block or a clearly labeled algorithm section. |
| Open Source Code | No | The paper does not contain any explicit statement about open-sourcing their code, nor does it provide a link to a code repository. |
| Open Datasets | No | The true regression function is given by f (x) = min(x, 1 x) with x Uniform[0, 1] and the observations are generated by the additive noise model yi = f (xi) + ϵi where ϵi N(0, σ2) and σ2 = 1 5. We generate N = 4098 sample points... |
| Dataset Splits | No | We generate N = 4098 sample points and use number of partitions m {2, 4, 8, 16, 32, 64, 128, 256, 512, 1024}... In the context of distributed kernel regression we divide the whole data D into m disjoint subset D = Sm ℓ=1 Dℓ. Without loss of generality we assume all data sets are of equal size n = N/m... |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details, such as library names with version numbers, needed to replicate the experiment. |
| Experiment Setup | Yes | According to Zhang et al. (2015), if the Soboleve space kernel KS is used, the theoretically optimal choice of the regularization parameter is λ = N 2 3 . When Guassian kernel is used, by Theorem 1 and Theorem 2, the optimal choice of λ should depend on the index r in the source condition, which, unfortunately, is unknown. By 0 < r < 1 2 we know the optimal choice should be N α with 1 2 < α < 1 and α = 2 3 seems an acceptable choice. So we will also use λ = N 2 3 for the Guassian kernel to make the first comparison between the four distributed kernel regression algorithms... |