Gradient Matching for Domain Generalization
Authors: Yuge Shi, Jeffrey Seely, Philip Torr, Siddharth N, Awni Hannun, Nicolas Usunier, Gabriel Synnaeve
ICLR 2022 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform experiments on the WILDS benchmark, which captures distribution shift in the real world, as well as the DOMAINBED benchmark that focuses more on syntheticto-real transfer. Our method produces competitive results on both benchmarks, demonstrating its effectiveness across a wide range of domain generalization tasks. |
| Researcher Affiliation | Collaboration | Yuge Shi University of Oxford EMAIL Jeffrey Seely Meta Reality Labs EMAIL Philip H.S. Torr University of Oxford EMAIL N. Siddharth The University of Edinburgh & The Alan Turing Institute EMAIL Awni Hannun Facebook AI Research EMAIL Nicolas Usunier Facebook AI Research EMAIL Gabriel Synnaeve Facebook AI Research EMAIL |
| Pseudocode | Yes | Algorithm 1 Fish. Algorithm 2 Direct optimization of IDGM. Algorithm 3 Black fonts denote steps used in both algorithms, colored fonts are steps unique to Fish or Reptile. Algorithm 4 Smoothed version of Fish, which allows to get approximate gradients for the general form of Equation (4). Algorithm 5 Function GIP. Algorithm 6 Algorithm of collecting gradient inner product g for Fish and ERM both before and after updates. |
| Open Source Code | Yes | Code is available at https://github.com/YugeTen/fish. |
| Open Datasets | Yes | We perform experiments on the WILDS benchmark (Koh et al., 2020)... as well as the DOMAINBED benchmark (Gulrajani and Lopez-Paz, 2020). |
| Dataset Splits | Yes | Table 5: Details of the 6 WILDS datasets we experimented on. Dataset... # Examples train val test... # Domains train val test |
| Hardware Specification | Yes | Our experiments can be replicated with 1500 GPU hours on NVIDIA V100. |
| Software Dependencies | No | The paper mentions optimizers (SGD, Adam) and model architectures (ResNet, DenseNet, DistilBERT) but does not provide specific version numbers for these or other software libraries or dependencies. |
| Experiment Setup | Yes | Table 13: Hyperparameters for ERM. We follow the hyperparameters used in WILDS benchmark. Table 14: Hyperparameters for Fish. |