Machine Unlearning via Simulated Oracle Matching
Authors: Kristian G Georgiev, Roy Rinberg, Sam Park, Shivam Garg, Andrew Ilyas, Aleksander Madry, Seth Neel
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we use a combination of existing evaluations and a new metric based on the KL-divergence to show that even in nonconvex settings, DMM achieves strong unlearning performance relative to existing algorithms. An added benefit of DMM is that it is a meta-algorithm, in the sense that future advances in data attribution translate directly into better unlearning algorithms, pointing to a clear direction for future progress in unlearning. |
| Researcher Affiliation | Collaboration | 1MIT EECS 2Harvard Business School 3Harvard SEAS 4Stanford CS 5Microsoft Research 6Stanford Statistics |
| Pseudocode | Yes | C PSEUDOCODE C.1 ORACLE MATCHING Algorithm C.1 Oracle Matching (OM) C.2 DATAMODEL DIRECT Algorithm C.2 DM-DIRECT C.3 DATAMODEL MATCHING Algorithm C.3 Datamodel Matching (DMM) |
| Open Source Code | Yes | Implementations for all methods are available at: bit.ly/unlearning-via-simulated-oracles |
| Open Datasets | Yes | We apply different approximate unlearning methods to trained DNNs to unlearn forget sets from CIFAR-10 and Image Net-Living-17. |
| Dataset Splits | Yes | On CIFAR-10, we use 9 different forget sets: sets 1,2,3 are random forget sets of sizes 10,100,1000 respectively; sets 4-9 correspond to semantically coherent subpopulations of examples (e.g., all dogs facing a similar direction) identified using clustering methods. On Image Net Living-17, we use three different forget sets: set 1 is random of size 500; sets 2 and 3 correspond to 200 examples from a certain subpopulation (corresponding to a single original Image Net class) within the Living-17 superclass. We re-train models on random 50% subsets of the full train dataset, and use between 1,000 and 20,000 models. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) are mentioned in the paper for the experimental setup. |
| Software Dependencies | No | The paper mentions a GitHub link to a Python script (https://github.com/wbaek/torchskeleton/blob/master/bin/dawnbench/cifar10.py) which likely uses a deep learning framework like PyTorch, but it does not specify any version numbers for Python or any libraries. |
| Experiment Setup | Yes | For CIFAR-10, we train Res Net-9 models5 for 24 epochs with SGD with a batch size of 512, momentum 0.9, and weight decay 5e 4. We set learning rate initially at 0.4, and a single-peak cosine schedule peaking at the 5th epoch. We use a momentum of 0.9 and a weight decay of 5e 4. For Image Net Living17 (Santurkar et al., 2020), we train Res Net-18 models for 25 epochs using SGD with a batch size of 1024, momentum 0.9, and weight decay 5e 4. Label smoothing is set to 0.1. |