Multi-Task Dynamical Systems
Authors: Alex Bird, Christopher K. I. Williams, Christopher Hawthorne
JMLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We apply the MTDS to motion-capture data of people walking in various styles using a multi-task recurrent neural network (RNN), and to patient drug-response data using a multi-task pharmacodynamic model. We provide two in-depth case studies: a multi-task RNN in Section 4 for a mocap data application, and a multi-task pharmacodynamic model in Section 5 for personalized drug response modelling. |
| Researcher Affiliation | Academia | Alex Bird EMAIL School of Informatics, University of Edinburgh, Edinburgh, EH8 9AB, UK and The Alan Turing Institute, London, NW1 2DB, UK; Christopher K. I. Williams EMAIL School of Informatics, University of Edinburgh, Edinburgh, EH8 9AB, UK and The Alan Turing Institute, London, NW1 2DB, UK; Christopher Hawthorne EMAIL Institute of Neurological Sciences, Queen Elizabeth University Hospital, Glasgow, G52 4TF, UK and Academic Unit of Anaesthesia, University of Glasgow, Glasgow, G31 2ER, UK |
| Pseudocode | Yes | Algorithm 1: Filtered inference via Iterated Ada IS. Result: Approximate posteriors {qt}T t=1 Inputs: y 1:T , u 1:T , φ, M, Mess, NAda IS, J q0 p(z) for t = 1 : T do ess 0 q0 prop qt 1 for n = 1 : NAda IS do for m = 1:M do zm qn 1 prop wm p(y 1:t | u 1:t,hφ(zm))p(z) qn 1 prop(zm) end wm wm PM ℓ=1 wℓ, m = 1, . . . , M qn prop Weighted EM {zm}M m=1, { wm}M m=1, J; init = qn 1 prop ess Effective Sample Size { wm}M m=1 if ess > Mess then break end end qt qn prop end |
| Open Source Code | Yes | An example implementation of the MTDS in Py Torch is provided at https://github.com/ornithos/pytorch-mtds-mocap, together with the data for the Mocap experiments. At the time of writing, the pharmacodynamic data is not yet publicly available, but a reference implementation of the model is available at https://gist.github.com/ornithos/71abb7349e91db633ce15971785bbae1. A Julia implementation of Ada IS for performing sequential inference in models with static latent variables (such as the MTDS) can be found at https://github.com/ornithos/Seq Adaptive IS. |
| Open Datasets | Yes | The data are obtained from Mason et al. (2018) which consists of planar walking and running motions in 8 styles recorded with a motion capture (mocap) suit by a single actor. ... The data were obtained from an anaesthesia study carried out at the Golden Jubilee National Hospital in Glasgow, Scotland, as described in Georgatzis et al. (2016). These consist of N = 40 time series of Caucasian patients; ... At the time of writing, the pharmacodynamic data is not yet publicly available |
| Dataset Splits | Yes | We perform six experiments which use between 28 to 213 frames per style (logarithmically spaced) for training, with sampling stratified carefully across major variations of all styles. ... In all cases, the test performance (MSE) is calculated from 4 held-out sequences from each style (64 frames each), and averaged over all styles. ... We consider a leave-one-out (LOO) procedure with eight folds, where each fold has a training set comprising 7 styles, and a test set comprising the held-out style. |
| Hardware Specification | Yes | The model was implemented in Py Torch (Paszke et al., 2017) and trained on an NVIDIA K80 GPU. |
| Software Dependencies | No | The model was implemented in Py Torch (Paszke et al., 2017) and trained on an NVIDIA K80 GPU. ... Even with the mature library Stan (Carpenter et al., 2016) we had to perform offline work to estimate the mass matrix of the sampler in order to avoid unstable chain dynamics and numerical problems. The paper mentions software names (PyTorch, Stan) but does not provide specific version numbers for these, or any other, key software components used in their experiments. |
| Experiment Setup | Yes | In detail, our proposed MTDS model is a 2 hidden-layer recurrent model where the first hidden layer is a 1024 unit GRU and the second hidden layer is a 128 unit MT-RNN, followed by a linear decoding layer. ... In our experiments, we use ℓ= 24 units for the bottleneck matrix H. ... The model was optimized for 20 000 iterations with Nbatch = 16 using the variational procedure in Section 2.2. ... We used Tenc = 64 frames. ... Table 7: Hyper-parameters of mocap models. η denotes the learning rate. MT-RNN Adam 3 10-5 1 10-3 1 10-2 |