ADMM for Nonconvex Optimization under Minimal Continuity Assumption
Authors: Ganzhao Yuan
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, some experiments on the sparse PCA problem are conducted to demonstrate the effectiveness of our approach. 1 5 EXPERIMENTS This section assesses the performance of IPDS-ADMM in solving the sparse PCA problem, as shown in Section 2. Compared Methods. We compare IPDS-ADMM against three state-of-the-art general-purpose algorithms that solve Problem (1): (i) the Subgradient method (Sub Grad) (Li et al., 2021; Davis & Drusvyatskiy, 2019), (ii) the Smoothing Proximal Gradient Method (SPGM) (B ohm & Wright, 2021), (iii) the Riemannian ADMM with fixed and large penalty (RADMM) (Li et al., 2022). Experimental Settings. All methods are implemented in MATLAB on an Intel 2.6 GHz CPU with 64 GB RAM. We incorporate a set of 8 datasets into our experiments, comprising both randomly generated and publicly available real-world data. Appendix Section E describes how to generate the data used in the experiments. For IPDS-ADMM, the relaxation parameter σ is set to be around the golden ratio 1.618, as suggested by (Li et al., 2016). Additionally, we set (ξ, p, δ, θ1, θ2) = (1/2, 1/3, 1/4, 1.01, 0.60). We denote ρ as the regularization parameter for sparse PCA model. The penalty parameter for RADMM is set to a reasonably large constant β = 100 ρ. We fix r = 20 and compare objective values for all methods after running T seconds, where T is reasonably large to ensure the proposed method converges. The corresponding MATLAB code is available on the author s research webpage. |
| Researcher Affiliation | Academia | Ganzhao Yuan Peng Cheng Laboratory, China EMAIL |
| Pseudocode | Yes | Algorithm 1: IPDS-ADMM: The Proposed Proximal Linearized ADMM for Problem (1). Choose suitable parameters {p, ξ, δ} and {σ, θ1, θ2} using Formula (5) or Formula (6). Initialize {x0, z0}. Choose β0 Ln/(δλ). for t from 0 to T do S1) IPDS Strategy: Set βt = β0(1 + ξtp), µt = 1/(λδβt). We define gt i xi G(xt+1 [1,i 1], xt i, xt [i+1,n], zt; βt). S2) xt+1 1 arg minx1 h1(x1) + x1 xt 1, gt 1 + θ1Lt 1 2 x1 xt 1 2 2 S3) xt+1 2 arg minx2 h2(x2) + x2 xt 2, gt 2 + θ1Lt 2 2 x2 xt 2 2 2 . . . S4) xt+1 n 1 arg minxn 1 hn 1(xn 1) + xn 1 xt n 1, gt n 1 + θ1Lt n 1 2 xn 1 xt n 1 2 2 S5) xt+1 n arg minxn hn(xn; µ) + xn xt n, gt n + θ2Lt n 2 xn xt n 2 2. It can be solved using Lemma 3.6 as xt+1 n = 1 1+µρ( xt+1 n + µρc), where xt+1 n = Proxn(c; µ + 1/ρ), µ = µt, ρ θ2Lt n, and c xt n gt n/ρ. S6) zt+1 = zt + σβt([Pn j=1 Ajxt+1 j ] b) end |
| Open Source Code | Yes | The corresponding MATLAB code is available on the author s research webpage. |
| Open Datasets | Yes | We incorporate a set of 8 datasets into our experiments, comprising both randomly generated and publicly available real-world data. Appendix Section E describes how to generate the data used in the experiments. ... The dataset names are as follows: TDT2md , sectormd , mnistmd , and randnmd . Here, randn(m, n) refers to a function that generates a standard Gaussian random matrix with dimensions m n. The matrix D R m d is constructed by randomly selecting m examples and d dimensions from the original real-world dataset (http://www.cad.zju.edu.cn/home/dengcai/ Data/Text Data.html,https://www.csie.ntu.edu.cn/ cjlin/libsvm/). We normalize each column of D to have a unit norm and center the data by subtracting the mean. |
| Dataset Splits | No | We incorporate a set of 8 datasets into our experiments, comprising both randomly generated and publicly available real-world data. Appendix Section E describes how to generate the data used in the experiments. ... The matrix D R m d is constructed by randomly selecting m examples and d dimensions from the original real-world dataset (http://www.cad.zju.edu.cn/home/dengcai/ Data/Text Data.html,https://www.csie.ntu.edu.cn/ cjlin/libsvm/). We normalize each column of D to have a unit norm and center the data by subtracting the mean. |
| Hardware Specification | Yes | All methods are implemented in MATLAB on an Intel 2.6 GHz CPU with 64 GB RAM. |
| Software Dependencies | No | All methods are implemented in MATLAB on an Intel 2.6 GHz CPU with 64 GB RAM. |
| Experiment Setup | Yes | For IPDS-ADMM, the relaxation parameter σ is set to be around the golden ratio 1.618, as suggested by (Li et al., 2016). Additionally, we set (ξ, p, δ, θ1, θ2) = (1/2, 1/3, 1/4, 1.01, 0.60). We denote ρ as the regularization parameter for sparse PCA model. The penalty parameter for RADMM is set to a reasonably large constant β = 100 ρ. We fix r = 20 and compare objective values for all methods after running T seconds, where T is reasonably large to ensure the proposed method converges. The corresponding MATLAB code is available on the author s research webpage. |