Causal Reinforcement Learning: A Survey
Authors: Zhihong Deng, Jing Jiang, Guodong Long, Chengqi Zhang
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this survey, we provide a comprehensive review of the literature in this domain. |
| Researcher Affiliation | Academia | Zhihong Deng EMAIL Australian Artificial Intelligence Institute, University of Technology Sydney Jing Jiang EMAIL Australian Artificial Intelligence Institute, University of Technology Sydney Guodong Long EMAIL Australian Artificial Intelligence Institute, University of Technology Sydney Chengqi Zhang EMAIL Australian Artificial Intelligence Institute, University of Technology Sydney |
| Pseudocode | No | The paper is a survey and describes various concepts and existing methods conceptually. It does not present any novel algorithms with pseudocode or algorithm blocks. |
| Open Source Code | No | The paper is a survey of causal reinforcement learning and does not describe a novel methodology or provide access to its own source code. It references various external projects and datasets, some of which may have open-source code (e.g., 'BARK-ML: https://github.com/bark-simulator/bark-ml', 'D4RL: https://github.com/Farama-Foundation/D4RL'), but this is not the code for the survey paper itself. |
| Open Datasets | Yes | MIMIC-III: https://physionet.org/content/mimiciii/1.4/. MIMIC-III (Johnson et al., 2016) is a large, freely-available database comprising deidentified health-related data associated with over forty thousand patients who stayed in critical care units of the Beth Israel Deaconess Medical Center between 2001 and 2012. Lu et al. (2020) use the code (https://github.com/aniruddhraghu/sepsisrl) provided by Raghu et al. (2017) for preprocessing the data and Bica et al. (2021b) open-source their code on https://github.com/vanderschaarlab/Invariant-Causal-Imitation-Learning/tree/main. Hi RID: https://physionet.org/content/hirid/1.1.1/. The Hi RID dataset (Hyland et al., 2020; Faltys et al., 2021), comprising data from over 34,000 patient admissions at the Bern University Hospital s Department of Intensive Care Medicine in Switzerland, offers a high-resolution collection of demographic, physiological, diagnostic, and treatment information. D4RL: https://github.com/Farama-Foundation/D4RL. D4RL is an open-source benchmark for offline RL. It includes several Open AI Gym benchmark tasks, such as the Hopper, Half Cheetah, and Walker environments. |
| Dataset Splits | No | As a survey paper, this document reviews existing literature and their experimental setups. It does not conduct its own experiments or specify dataset splits for novel research. While it mentions the use of various datasets and environments in other works (e.g., MIMIC-III, HiRID, Open AI Gym tasks), it does not provide specific training/test/validation splits for any experimental setup that it would have conducted. |
| Hardware Specification | No | This paper is a survey of causal reinforcement learning. It does not describe any original experimental work performed by the authors, and therefore does not specify hardware used for experiments. |
| Software Dependencies | No | This paper is a survey of causal reinforcement learning. It does not describe any original experimental work performed by the authors, and therefore does not specify software dependencies with version numbers for experiments. |
| Experiment Setup | No | This paper is a survey of causal reinforcement learning. It does not describe any original experimental work performed by the authors, and therefore does not provide specific details about experimental setup, hyperparameters, or training configurations. |