Tensor Train Decomposition on TensorFlow (T3F)
Authors: Alexander Novikov, Pavel Izmailov, Valentin Khrulkov, Michael Figurnov, Ivan Oseledets
JMLR 2020 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4. Benchmarking We benchmark the basic functionality of T3F on CPU and GPU and compare its performance against an alternative library TTPY. To reproduce the benchmark on your hardware, see docs/benchmark folder in the T3F library. For benchmarking, we generated a batch of 100 random TT-matrices of sizes 1010 1010 (so d = 10 and the TT-representation consists of 10 TT-cores) of TT-rank 10 and a batch of 100 random TT-vectors of size 1010 1. We benchmarked the matrix-by-vector multiplication ( matvec ), matrix-by-matrix multiplication ( matmul ), computing the Frobenius norm ( norm ), and computing the Gram matrix of 1 or 100 TT-vectors. The results are reported in Tbl. 1. We report that T3F is faster than TTPY for most operation and that batch processing and GPU acceleration yield significant speedups for some operations (Tbl. 1). |
| Researcher Affiliation | Academia | 1 National Research University Higher School of Economics, Moscow, Russia 2 Institute of Numerical Mathematics RAS, Moscow, Russia 3 Cornell University, Ithaca NY, USA 4 Skolkovo Institute of Science and Technology, Moscow, Russia |
| Pseudocode | No | The paper describes the implementation details and functions of the T3F library, but it does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The library is released1 under MIT license and is distributed as a Py PI package2 to simplify the installation process. 1https://github.com/Bihaqo/t3f 2https://pypi.python.org/pypi/t3f |
| Open Datasets | No | The paper uses generated random TT-matrices and TT-vectors for benchmarking. It does not provide access information for a publicly available or open dataset. |
| Dataset Splits | No | The paper uses generated random TT-matrices and TT-vectors for benchmarking. It does not define train/test/validation splits for any dataset. |
| Hardware Specification | Yes | The comparison is made on an NVIDIA DGX-1 station with Tesla V100 GPUs (using only 1 GPU at a time) in double precision. |
| Software Dependencies | No | The paper states that T3F is based on Tensor Flow but does not specify a version number for TensorFlow or any other software dependency. |
| Experiment Setup | No | The paper describes the generation of random TT-matrices and TT-vectors for benchmarking, including their sizes and TT-rank. However, it does not specify concrete hyperparameters, training configurations, or system-level settings typically found in experimental setups for machine learning models (e.g., learning rate, batch size, optimizer settings), as the paper focuses on a library rather than a specific trained model. |