Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method
Authors: Andi Han, Pierre-Louis Poirion, Akiko Takeda
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments verify the benefits of the proposed method, across a wide variety of problems. 6. Experiments This section conducts experiments to verify the efficacy of the proposed method. |
| Researcher Affiliation | Academia | 1RIKEN AIP 2University of Sydney 3University of Tokyo. Correspondence to: Andi Han <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 RSDM |
| Open Source Code | Yes | All experiments are implemented in Pytorch and run on a single RTX4060 GPU. The code is available on https://github.com/ andyjm3/RSDM. |
| Open Datasets | Yes | We optimize neural networks (orthogonal FFN and orthogonal Vi T) to classify MNIST (Le Cun et al., 1998) and CIFAR10 (Krizhevsky et al., 2009) images. |
| Dataset Splits | Yes | We optimize neural networks (orthogonal FFN and orthogonal Vi T) to classify MNIST (Le Cun et al., 1998) and CIFAR10 (Krizhevsky et al., 2009) images. |
| Hardware Specification | Yes | All experiments are implemented in Pytorch and run on a single RTX4060 GPU. |
| Software Dependencies | No | All experiments are implemented in Pytorch and run on a single RTX4060 GPU. |
| Experiment Setup | Yes | For all experiments, we tune the learning rate in the range of [0.01, 0.05, 0.1, 0.5, 1.0, 1.5, 2.0]. For the infeasible methods, we tune the regularization parameter in the range of [0.1, 0.5, 1.0, 1.5, 2.0]. For optimization, we employ RGD and RSDM with a batch size of 16. We set learning rate for unconstrained parameters to be 0.1 and only tune the learning rate for the orthogonal parameters. |