Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent
Authors: Santhosh Karnik, Anna Veselovska, Mark Iwen, Felix Krahmer
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theoretical findings are illustrated in an extensive set of numerical simulations show-casing the dynamics predicted by our theory as well as the crucial role of using a small random initialization. To verify our theoretical findings, we set multiple numerical tests: from showing two phases of the gradient descent algorithm to demonstrating the advantages of overparametrization. These experimental results showcase not only the implicit regularization for the gradient descent algorithm toward low-tubal-rank tensors but also demonstrate the firmness of our theoretical findings. |
| Researcher Affiliation | Academia | 1Department of Mathematics, Northeastern University, Boston, USA 2Department of Mathematics and Munich Data Science Institute, Technical University of Munich, Munich, Germany 3Munich Center for Machine Learning, Munich, Germany 4Department of Mathematics, Michigan State University, East Lansing, USA 5Department of Computational Mathematics Science and Engineering, Michigan State University, East Lansing, USA. |
| Pseudocode | No | The paper describes mathematical proofs and derivations, along with numerical experiments. It does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | The corresponding code is available in our Git Hub repository, https://github.com/AnnaVeselovska_UA/tubal-tensor-implicit-reg-GD.git. |
| Open Datasets | No | We generate the ground truth tensor T Rn r k with tubal rank r by T = X X , where the entries of X Rn r k are i.i.d. sampled from a Gaussian distribution N(0, 1), and then X is normalized. The entries of measurement tensor Ai are i.i.d. sampled from a Gaussian distribution N(0, 1/m). |
| Dataset Splits | No | The paper uses synthetically generated data for its experiments, rather than a pre-existing dataset that would require specific training/test/validation splits. |
| Hardware Specification | Yes | Our experiments were conducted on a Mac Book Pro equipped with an Apple M1 processor and 16GB of memory, using MATLAB 2023a software. |
| Software Dependencies | Yes | Our experiments were conducted on a Mac Book Pro equipped with an Apple M1 processor and 16GB of memory, using MATLAB 2023a software. |
| Experiment Setup | Yes | For all the experiments, we set the dimensions to n = 10, k = 4, r = 3, the learning rate ยต = 10^-5, and the number of measurements m = 254. |