Towards Continual Reinforcement Learning: A Review and Perspectives
Authors: Khimya Khetarpal, Matthew Riemer, Irina Rish, Doina Precup
JAIR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this article, we aim to provide a literature review of different formulations and approaches to continual reinforcement learning (RL), also known as lifelong or non-stationary RL. We begin by discussing our perspective on why RL is a natural fit for studying continual learning. We then provide a taxonomy of different continual RL formulations by mathematically characterizing two key properties of non-stationarity, namely, the scope and driver non-stationarity. This offers a unified view of various formulations. Next, we review and present a taxonomy of continual RL approaches. |
| Researcher Affiliation | Collaboration | Khimya Khetarpal EMAIL Mila, Mc Gill University Matthew Riemer EMAIL Mila, Universit e de Montr eal, IBM Research Irina Rish EMAIL Mila, Universit e de Montr eal Doina Precup EMAIL Mila, Mc Gill University, Deep Mind |
| Pseudocode | No | The paper is a literature review and taxonomy of continual RL. It describes various approaches conceptually and mathematically, but does not present any algorithms or pseudocode in a structured format. |
| Open Source Code | No | The paper is a literature review and taxonomy. It does not describe any specific new methodology for which open-source code would be provided. There are no statements regarding the release of code for this work. |
| Open Datasets | No | The paper is a review and does not present experimental results using specific datasets. It discusses various benchmarks such as 'Open AI Gym', 'Arcade Learning Environment (ALE)', 'Jelly Bean World', 'Causal World', and 'Meta-world' in the context of evaluating continual RL agents, but it does not use them for experiments within this work. |
| Dataset Splits | No | This paper is a literature review and taxonomy of continual reinforcement learning. It does not describe any experiments that would require specifying training, validation, or test dataset splits. |
| Hardware Specification | No | This paper is a literature review and taxonomy of continual reinforcement learning. It does not describe any experiments that would require specific hardware for execution. |
| Software Dependencies | No | This paper is a literature review and taxonomy of continual reinforcement learning. It does not describe any experiments or implementations that would require specific software dependencies with version numbers. |
| Experiment Setup | No | This paper is a literature review and taxonomy of continual reinforcement learning. It does not describe any experiments, training procedures, or hyperparameter settings in its content. |