CarbonSense: A Multimodal Dataset and Baseline for Carbon Flux Modelling
Authors: Matthew Fortier, Mats L. Richter, Oliver Sonnentag, Christopher Pal
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide a baseline model using a current state-of-the-art DDCFM approach and a novel transformer based model. Our experiments illustrate the potential gains that multimodal deep learning techniques can bring to this domain. By providing these resources, we aim to lower the barrier to entry for other deep learning researchers to develop new models and drive new advances in carbon flux modelling. |
| Researcher Affiliation | Collaboration | Matthew Fortier Mila Quebec AI Institute EMAIL Mats L. Richter Service Now Research Oliver Sonnentag Département de géographie Université de Montréal Chris Pal Mila Quebec AI Institute Polytechnique Montréal Canada CIFAR AI Chair |
| Pseudocode | No | The paper describes the Eco Perceiver architecture using figures (e.g., Figure 4, Figure 5, Figure 6) and textual descriptions of its components and processes. It does not contain explicit pseudocode blocks or algorithms labeled as such. |
| Open Source Code | Yes | We offer Carbon Sense as a finished dataset but also provide the raw data. The full pipeline code is available so that researchers can run and modify it freely. |
| Open Datasets | Yes | To address this gap, we present Carbon Sense, the first machine learning-ready dataset for DDCFM. Carbon Sense integrates measured carbon fluxes, meteorological predictors, and satellite imagery from 385 locations across the globe, offering comprehensive coverage and facilitating robust model training. All meteorological data was aggregated from major EC data networks, including FLUXNET 2015 [14], the Integrated Carbon Observation System (ICOS) 2023 release [22], ICOS Warm Winter release [23], and Ameriflux 2023 release [24]. These source datasets were chosen due to their use of the ONEFlux processing pipeline [14], ensuring standardized coding and units. Geospatial data in Carbon Sense are sourced from MODIS products. Specifically, we utilize the seven spectral bands from the MCD43A4 product [19], as well as the water and snow cover bands from MCD43A2 [20]. |
| Dataset Splits | Yes | EC stations were randomly divided into train and test sets based on their IGBP ecosystem classification (IGBP type). We mostly refer to IGBP types by their acronyms for brevity, but a list of IGBP types with expanded names is found in Appendix A. The number of sites in the test set were determined with min(5, 0.2 num_sites ). This provided between 1 and 5 sites per IGBP type as shown in Table 1. The train sites were further divided into train and validation splits at a 0.8 / 0.2 ratio respectively. |
| Hardware Specification | Yes | Eco Perceiver experiments were each run on 4 A100 GPUs using dataset parallelization. ... XGBoost experiments were run on CPU nodes. |
| Software Dependencies | No | The paper mentions software like PyTorch and XGBoost but does not provide specific version numbers for these or other libraries used. |
| Experiment Setup | Yes | Eco Perceiver experiments were each run on 4 A100 GPUs using dataset parallelization. The train sites were further divided into train and validation splits at a 0.8 / 0.2 ratio respectively. We used the Adam W optimizer [30] with a learning rate of 8e-5 and a batch size of 4096. A single warm-up epoch was performed followed by a cosine annealing learning rate schedule over 20 epochs, but all experiments converged between 6 and 13 epochs. We set our latent hidden size to 128, our input embedding size to 16, and the number of Fourier encoding frequencies to 12. This gave a total input hidden size of 40. Our context window is 32, meaning our model sees the previous 32 hours of observations. We use 8 WCA blocks followed by 4 CSA blocks. We set our observational dropout at 0.3 and use causal masking in all self-attention blocks. Table 7: XGBoost Hyperparameters (learning_rate 0.1, alpha 0.1, gamma 0.4, lambda 0.0, max_depth 9, min_child_weight 9, n_estimators 150, subsample 0.7, scale_pos_weight 0.5, colsample_bytree 0.7, colsample_bylevel 0.8) |