Composite Self-Concordant Minimization
Authors: Quoc Tran-Dinh, Anastasios Kyrillidis, Volkan Cevher
JMLR 2015 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We describe concrete algorithmic instances of our framework for several interesting applications and demonstrate them numerically on both synthetic and real data. Keywords: proximal-gradient/Newton method, composite minimization, self-concordance, sparse convex optimization, graph learning 5. Numerical Experiments In this section, we illustrate our optimization framework via numerical experiments on the variants discussed in Section 4. We only focus on proximal gradient and Newton variants and encourage the interested reader to try out the quasi-Newton variants for their own applications. |
| Researcher Affiliation | Academia | Quoc Tran-Dinh EMAIL Anastasios Kyrillidis EMAIL Volkan Cevher EMAIL Laboratory for Information and Inference Systems (LIONS) Ecole Polytechnique F ed erale de Lausanne (EPFL) CH1015-Lausanne, Switzerland |
| Pseudocode | Yes | Algorithm 1 (Proximal-Newton algorithm) Inputs: x0 dom(F), tolerance ε > 0. Initialization: Select a constant σ (0, (5 17) 4 ], e.g., σ := 0.2. for k = 0 to Kmax do 1. Compute the proximal-Newton search direction dk n as in (19). 2. Compute λk := dk n xk. 3. if λk > σ then xk+1 := xk + αkdk n, where αk := (1 + λk) 1. 4. elseif λk > ε then xk+1 := xk + dk n. 5. else terminate. end for |
| Open Source Code | Yes | We also provide MATLAB implementations of the examples in this section as a software package (SCOPT) at http://lions.epfl.ch/software. |
| Open Datasets | Yes | The first problem is a synthetic examples of size p = 10, where the data is generated as in (Kyrillidis and Cevher, 2013). We run this test for 10 times and report computational primitives in average. Three remaining problems are based on real data from http://ima.umn.edu/~maxxa007/send_SICS/, where the regularization parameters are chosen as the standard values (cf., Tran-Dinh et al. (2013a); Lee et al. (2012); Hsieh et al. (2011)). |
| Dataset Splits | No | The paper mentions using synthetic examples and real data, and discusses specific problem instances like Lymph, Estrogen, house, and cameraman, but does not provide explicit details about how these datasets were split into training, validation, or test sets (e.g., percentages or sample counts). |
| Hardware Specification | Yes | All the tests are performed in MATLAB 2011b running on a PC Intel Xeon X5690 at 3.47GHz per core with 94Gb RAM.6 (i) p FISTA and d FISTA: in these cases, we use the FISTA algorithm (Beck and Teboulle, 2009a) for solving the primal (37) and the dual subproblem (39). Moreover, to speedup the computations, we further run these methods on the GPU [NVIDIA Quadro 4000]. |
| Software Dependencies | Yes | All the tests are performed in MATLAB 2011b running on a PC Intel Xeon X5690 at 3.47GHz per core with 94Gb RAM.6 |
| Experiment Setup | Yes | We terminate the proximal Newton scheme if λk 10 6. The numerical results are summarized in Table 2. Here, #iter denotes the (average) number of iterations, #chol represents the (average) number of Cholesky decompositions and #Mm is the (average) number of matrix-matrix multiplications. Image ρ 10 5 #iteration CPU time [s] AC F k min F house 1.0 116 256 500 27.45 56.95 1658.00 60 29 -10718352.93 0.31 0.70 (256 256) 2.5 92 244 500 18.18 50.26 1431.94 79 28 -10711758.80 3.20 3.32 barbara 1.0 200 324 500 46.92 77.77 1204.36 26 15 -7388497.47 0.05 0.30 (256 256) 2.5 164 268 500 36.45 67.98 1620.95 44 24 -7377424.50 1.90 2.02 |