Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning
Authors: Kaiyi Ji, Junjie Yang, Yingbin Liang
JMLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we develop a new theoretical framework to provide such convergence guarantee for two types of objective functions that are of interest in practice... For both cases, we characterize the convergence rate and the computational complexity to attain an ̖-accurate solution for multi-step MAML in the general nonconvex setting. In particular, our results suggest that an inner-stage stepsize needs to be chosen inversely proportional to the number N of inner-stage steps in order for N-step MAML to have guaranteed convergence. From the technical perspective, we develop novel techniques to deal with the nested structure of the meta gradient for multi-step MAML, which can be of independent interest. Keywords: Computational complexity, convergence rate, nite-sum, meta-learning, multi-step MAML, nonconvex, resampling. |
| Researcher Affiliation | Academia | Kaiyi Ji EMAIL Department of Electrical and Computer Engineering The Ohio State University Columbus, OH 98195-4322, USA; Junjie Yang EMAIL Department of Electrical and Computer Engineering The Ohio State University Columbus, OH 98195-4322, USA; Yingbin Liang EMAIL Department of Electrical and Computer Engineering The Ohio State University Columbus, OH 98195-4322, USA |
| Pseudocode | Yes | Algorithm 1 Multi-step MAML in the resampling case; Algorithm 2 Multi-step MAML in the nite-sum case |
| Open Source Code | No | The paper provides a license for the paper itself ('License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v23/20-720.html.') but does not include any statements or links regarding the release of source code for the methodology described. |
| Open Datasets | No | The paper is theoretical and focuses on convergence guarantees. While Appendix A provides 'Examples for Two Types of Objective Functions' including 'RL Example for Resampling Case' and 'Classication Example for Finite-Sum Case', these are generic problem descriptions and not specific, accessible datasets with concrete access information like links, DOIs, or citations to established benchmarks. |
| Dataset Splits | No | The paper is theoretical and does not conduct experiments with specific datasets. Therefore, it does not describe training/test/validation dataset splits or any other data partitioning details for reproducibility. |
| Hardware Specification | No | This is a theoretical paper providing convergence analysis. It does not describe any experimental setup or hardware used for running experiments. |
| Software Dependencies | No | This is a theoretical paper. It does not mention any specific software components or libraries with version numbers that would be required to reproduce experimental results. |
| Experiment Setup | No | This is a theoretical paper focusing on convergence analysis. It does not describe specific experimental setups, hyperparameters, or training configurations for empirical validation. Parameters like 'inner stepsize ̖' and 'meta stepsize ̖k' are part of the theoretical analysis, not an experimental setup. |