Multi-Objective Markov Decision Processes for Data-Driven Decision Support
Authors: Daniel J. Lizotte, Eric B. Laber
JMLR 2016 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we demonstrate an application of our method using data from the Clinical Antipsychotic Trials of Intervention Effectiveness and show that our approach offers decision-makers increased choice by a larger class of optimal policies. Keywords: multi-objective optimization, reinforcement learning, Markov decision processes, clinical decision support, evidence-based medicine ... We demonstrate the use of our algorithm on the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) ... Empirical Example: CATIE |
| Researcher Affiliation | Academia | Daniel J. Lizotte EMAIL Department of Computer Science, Department of Epidemiology & Biostatistics The University of Western Ontario 1151 Richmond Street London, ON N6A 3K7 Canada Eric B. Laber EMAIL Department of Statistics North Carolina State University Raliegh, NC 27695 USA |
| Pseudocode | Yes | Algorithm 1 Non-deterministic fitted-Q |
| Open Source Code | No | The paper does not provide any explicit links to source code repositories, nor does it state that the code is available in supplementary materials or upon request. There is no clear statement about releasing the code for the described methodology. |
| Open Datasets | Yes | We illustrate the output of non-deterministic fitted-Q using data from the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study. ... Data used in the preparation of this article were obtained from the limited access data sets distributed from the NIH-supported Clinical Antipsychotic Trials of Intervention Effectiveness in Schizophrenia (CATIE-Sz). The study was supported by NIMH Contract N01MH90001 to the University of North Carolina at Chapel Hill. The Clinical Trials.gov identifier is NCT00014001. |
| Dataset Splits | No | The paper describes the CATIE study, including the number of patients (n = 1460) and the phases of treatment, but it does not specify any training, validation, or test dataset splits for the experiments conducted in the paper. |
| Hardware Specification | Yes | Finding the φ-consistent policies took less than one minute on an Intel Core i7 at 3.4 GHz using Python and CPLEX. |
| Software Dependencies | Yes | CPLEX. ILOG CPLEX Optimizer. http://www-01.ibm.com/software/integration/optimization/cplex-optimizer/, 2012. ... Finding the φ-consistent policies took less than one minute on an Intel Core i7 at 3.4 GHz using Python and CPLEX. |
| Experiment Setup | Yes | We will use ordinary least squares to learn Q functions for two basis rewards. ... We present our method assuming that ˆw Trds are found by least squares regression, but one could for example add an L1 penalty, or use support vector regression (Hastie et al., 2001). Furthermore, unlike previous work by Lizotte et al. (2012), any Generalized Linear Model (GLM) with a monotonic increasing link function (e.g. logistic regression, Poisson regression, and so on) can also be used (Cook and Weisberg, 1999). ... For our state space, we use the patient s most recently recorded PANSS score... We also include their most recent BMI, and several baseline characteristics. ... The feature vectors we use for Stage 2 Efficacy patients are given by φEFFps2, a2q r1, 1TD, 1EX, 1ST1, 1ST2, 1ST3, 1ST4, s2:P, s2:B, 1a2 , s2:P 1a2 , s2:B 1a2 s. ... Figure 7 shows the NDP learned for Phase 1 using our algorithm with Strong Practical Domination (ε1 ε2 = 2.5) and ΠD ă, and actions that receive a warning according to Practical Domination have been removed. |