Deep Kernel Learning of Nonlinear Latent Force Models
Authors: Jacob Moss, Jeremy England, Pietro Lio
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we demonstrate the ability to extrapolate a solution operator trained on simulations to real experimental datasets, as well as scaling to large datasets. 4 Experiments In this section we investigate the performance of DKLFM on two ODE-based LFMs and one PDE-based LFM. Given that this is the first multi-task model for latent force models, we analyse the performance on real, experimentally-derived datasets not in the synthetic training distribution. We compare our approach with two models from the literature on solving latent force problems. Table 1: Comparison to baseline models for the transcriptional regulation ODE and the the reaction diffusion PDE. |
| Researcher Affiliation | Collaboration | Jacob Moss EMAIL Computer Lab University of Cambridge Jeremy England GSK Petach Tikva, Israel Pietro Lió Computer Lab University of Cambridge |
| Pseudocode | Yes | A.1 Pseudocode Algorithm 1 Training procedure of DKLFM for each task n do rn emb(xn, yn) Compute representations zn = rn xn Concatenate inputs Kff = κ (NNc(NNf(zn)), NNc(NNf(z n))) Latent force covariance Kfh = κ (NNc(NNf(zn)), NNc(NNh(z n))) Cross-function covariance Khh = κ (NNc(NNh(zn)), NNc(NNh(z n))) Output covariance µh|f = µh + Khf K 1 ff f Kh|f = Khh Khf K 1 ff Kfh p(y, f) = N(f | µf, Kff)N(y | µh|f, Kh|f + σ2I) backpropagate log p(y, f) end for Algorithm 2 DKLFM inference for each task n do rn emb(xn, yn) Compute representations zn = rn xn Concatenate inputs r n emb(x n, y n) z n = rn x n Khh = κ (NNc(NNh(zn)), NNc(NNh(z n))) Output covariance Ky h = κ (NNc(NNh(z n)), NNc(NNh(z n))) Output covariance Kf h = κ (NNc(NNf(z n)), NNc(NNh(z n))) Cross-function covariance µf |y = µf + Kf h K 1 hh y Kf |y = Kf f Kf h K 1 hh Khf µy |y = µy + Ky h K 1 hh y Ky |y = Ky y Ky h K 1 hh Khy end for return N(µf |y, Kf |y), N(µy |y, Ky |y) |
| Open Source Code | No | The paper does not explicitly state that source code for the methodology is openly available, nor does it provide a link to a code repository. It mentions using third-party tools like Transformer and Fourier neural operator, but not the authors' implementation. |
| Open Datasets | Yes | We validate additionally on experimentally-derived data from Barenco et al. (2006), where cancer cells were subject to ionising radiation and the concentration of m RNA was measured via microarray at different timepoints. From an empirical inspection of the gap gene dataset from Becker et al. (2013), we uniformly sampled production rates in the range [0.2, 1.0], decay rates in the range [0.01, 0.4], and diffusion rates in the range [0.001, 0.1]. |
| Dataset Splits | Yes | We generate 500 instances and split into training, validation, and test tasks. We generate 448 tasks in this fashion, taking less than two hours on an AMD Ryzen 5600x, of which 384 are used for training. In Figure 6, we confirm our hypothesis by plotting the MSE versus dataset size for our model, and horizontal lines are the mean MSE for Alfi over a subset of 64 tasks. |
| Hardware Specification | Yes | Training was on an NVIDIA Ge Force RTX 4090 GPU. Due to the differentiable PDE-solving package used by Alfi not being GPU-compatible, the PDE tasks were fit on an AMD Ryzen 5600x CPU. We generate 448 tasks in this fashion, taking less than two hours on an AMD Ryzen 5600x |
| Software Dependencies | No | The paper mentions a "differentiable PDE-solving package used by Alfi" and a "4th-order Runge-Kutta solver", but does not provide specific version numbers for these or any other software dependencies like programming languages or libraries. |
| Experiment Setup | Yes | We start by sampling parameters for Equation 4 from an empirical distribution of parameters learnt by running the Alfi (Moss et al., 2021) latent force inference package on the p53 network of genes experimentally measured by Barenco et al. (2006). Next, the latent force is sampled from a GP prior with RBF kernel... We simulated a dataset of Lotka-Volterra solutions corresponding to different sampled rates, α, β U(0.5, 1.5), using a 4th-order Runge-Kutta solver. From an empirical inspection of the gap gene dataset from Becker et al. (2013), we uniformly sampled production rates in the range [0.2, 1.0], decay rates in the range [0.01, 0.4], and diffusion rates in the range [0.001, 0.1]. For the latent force, we sampled the two lengthscales (corresponding to spatial and temporal dimensions) in the range [0.1, 0.4] since both dimensions are normalised to [0, 1]. The model was trained with a 21 21 spatiotemporal grid. At prediction time, 40 40 grid was used to illustrate the super-resolution capability. |