TorchOpt: An Efficient Library for Differentiable Optimization
Authors: Jie Ren*, Xidong Feng*, Bo Liu*, Xuehai Pan*, Yao Fu, Luo Mai, Yaodong Yang
JMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results demonstrate that Torch Opt achieves a 5.2 training time speedup in a cluster. |
| Researcher Affiliation | Academia | 1 School of Informatics, University of Edinburgh, Edinburgh, United Kingdom 2 Department of Computer Science, University College London, London, United Kingdom 3 School of Computer Science, Peking University, Beijing, China 4 ECRC, KAUST, Thuwal, Saudi Arabia 5 Institute for AI, Peking University, Beijing, China |
| Pseudocode | No | The paper includes 'Listing 1: Torch Opt code snippet for explicit gradient.', 'Listing 2: Torch Opt code snippet for implicit gradient.', and 'Listing 3: Torch Opt code snippet for zero-order diļ¬erentiation.'. These are referred to as 'code snippets' and illustrate usage, rather than being formal pseudocode or algorithm blocks with structured steps labeled as such. |
| Open Source Code | Yes | Torch Opt is open-sourced at https://github.com/metaopt/torchopt and has become a Py Torch Ecosystem project. |
| Open Datasets | Yes | In Fig. 7, we show the training accuracy and wall time comparison on the MAML Omniglot example. |
| Dataset Splits | No | The paper mentions using the 'MAML Omniglot example' for experiments but does not provide specific details on how the dataset was split into training, validation, or test sets, nor does it refer to standard splits with citations. |
| Hardware Specification | No | The paper discusses 'CPU-accelerated' and 'GPU-accelerated' optimizers and refers to experiments on '8 GPUs' in Fig. 7. However, it does not specify the exact models or types of CPUs or GPUs used (e.g., 'NVIDIA A100', 'Intel Xeon'). |
| Software Dependencies | No | The paper states that Torch Opt is a 'Py Torch-based diļ¬erentiable optimization library' and mentions 'Py Torch libraries'. It also refers to 'Python' in the context of `pybind11`. However, no specific version numbers for PyTorch, Python, or any other critical software dependencies are provided. |
| Experiment Setup | No | The paper includes code snippets showing the use of `torchopt.adam()` and `torchopt.MetaAdam()`, and discusses various differentiation modes. However, it does not provide specific hyperparameter values (e.g., learning rate, batch size, number of epochs, optimizer settings like beta values) for the experiments conducted or demonstrated in the text. |