Localisation of Regularised and Multiview Support Vector Machine Learning
Authors: Aurelian Gheondea, Cankat Tilki
JMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We prove some representer theorems for a localised version of a semisupervised, manifold regularised and multiview support vector machine learning problem introduced by H.Q. Minh, L. Bazzani, and V. Murino, Journal of Machine Learning Research, 17(2016) 1 72, that involves operator valued positive semidefinite kernels and their reproducing kernel Hilbert spaces. The results concern general cases when convex or nonconvex loss functions and finite or infinite dimensional underlying Hilbert spaces are considered. We show that the general framework allows infinite dimensional Hilbert spaces and nonconvex loss functions for some special cases, in particular in case the loss functions are Gˆateaux differentiable. Detailed calculations are provided for the exponential least squares loss functions that lead to systems of partially nonlinear equations for which some Newton s approximation methods based on the interior point method can be used. Some numerical experiments are performed on a toy model that illustrate the tractability of the methods that we propose. |
| Researcher Affiliation | Academia | Aurelian Gheondea EMAIL Institute of Mathematics of the Romanian Academy 21 Calea Grivit ei 010702 Bucharest, Romania and Department of Mathematics Bilkent University 06800 Bilkent, Ankara, Turkey EMAIL Cankat Tilki EMAIL Department of Mathematics and Division of Computational Modeling and Data Analytics Virginia Polytechnic Institute and State University Blacksburg Virginia, 24061 U.S.A. |
| Pseudocode | Yes | Algorithm 1 Numerical Implementation Require: xi X1 X2 R2, σ, α > 0, γI, γA > 0. 1: D {x1, x2, . . . , x6} 2: Construct M in (3.40) and K in (3.48) using data D 3: Compute δ in (3.49) 4: Compute the LHS L on the cube [ δ, δ]9 5: for each element a0 L do 6: Solve the minimisation problem (3.29) and (3.30) on the cube [ δ, δ]9 with initial condition a0 using the function fmincon. 7: if a0 is admissible in the sense of Remark 20 then 8: a a0 9: end if 10: end for 11: return a |
| Open Source Code | No | The text does not include an unambiguous sentence stating the authors are releasing their code or a direct link to a source-code repository for their methodology. |
| Open Datasets | No | In this subsection we provide a toy model for the localised version of the regularised machine learning problem in case the loss function is the exponential least square function as in Subsection 3.4 and test an algorithm following the discussion of the numerical methods as in Subsection 3.5. To this end, let X = X1 X2, where X1 := {(α1, α2) | 0.25 α1 1, 0.25 α2 1}, X2 := {(α1, α2) | 1 α1 0.25, 0.25 α2 1}. In the following we use the notation as in Subsection 2.2. We consider x1 X1 and x2 X2 randomly selected and let the labels y1 Yx1 = R and y2 Yx2 = R2 be randomly selected. Also, let x3, x4 X1 \ {x1}, x3 = x4, and x5, x6 X2 \ {x2}, x5 = x6, randomly selected as well, be unlabeled points. |
| Dataset Splits | No | We consider x1 X1 and x2 X2 randomly selected and let the labels y1 Yx1 = R and y2 Yx2 = R2 be randomly selected. Also, let x3, x4 X1 \ {x1}, x3 = x4, and x5, x6 X2 \ {x2}, x5 = x6, randomly selected as well, be unlabeled points. In particular, l = 2 and u = 4. |
| Hardware Specification | No | The code runs efficiently on a Mac laptop, for this example it takes only less than a minute, but for other combinations of coefficients it may be five or more minutes. |
| Software Dependencies | No | An essential part of the algorithm is the use of the function fmincon of MATLAB that uses the method of interior points for constrained optimisation, see Byrd et al. (1999), Byrd et al. (2000), and Waltz et al. (2006). |
| Experiment Setup | Yes | In this example we used γA = 0.25, γI = 10, σ = 0.1 and α = 10, randomly generated the data x and y and returned the result a. The implicit optimality tolerance is ϵ = 10 6. |