Adversarial Machine Unlearning
Authors: Zonglin Di, Sixie Yu, Yevgeniy Vorobeychik, Yang Liu
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present empirical results to demonstrate the effectiveness of the proposed approach for machine unlearning. The code is available at https://github.com/UCSC-REAL/SG-Unlearn. ... 6 EXPERIMENTS We conduct experiments on both computer vision (CV) and natural language processing (NLP) datasets. For the CV tasks, we use the widely recognized image classification datasets CIFAR-10, CIFAR-100, and SVHN (Krizhevsky et al., 2009; Netzer et al., 2011). ... The experimental results for random forgetting and class-wise forgetting are presented in Section 6.3.1 and 6.3.2, respectively. We consider retrain as the gold standard for evaluating unlearning algorithms: the closer to the metrics of retrain the more effective the algorithm. We highlight the closest metrics to retrain in bold. |
| Researcher Affiliation | Collaboration | Zonglin Di1 , Sixie Yu2 , Yevgeniy Vorobeychik3, Yang Liu1 1University of California, Santa Cruz, 2Stellar Cyber, Inc., 3Washington University in St. Louis 1EMAIL, EMAIL EMAIL |
| Pseudocode | Yes | The pseudocode for this process is provided in Algorithm 1. This algorithm has a time complexity of O(n3), where big-O notations inherently describes an upper bound and n denotes the size of the attacker s optimization problem (i.e., the number of variables and/or constraints). |
| Open Source Code | Yes | The code is available at https://github.com/UCSC-REAL/SG-Unlearn. |
| Open Datasets | Yes | For the CV tasks, we use the widely recognized image classification datasets CIFAR-10, CIFAR-100, and SVHN (Krizhevsky et al., 2009; Netzer et al., 2011). ... For NLP tasks, we assess performance on the 20 Newsgroups dataset, leveraging the BERT (Devlin, 2018) model. ... The experiment results of Tiny Image Net (Le & Yang, 2015) and Celeb A (Liu et al., 2018) are given in Table 7 and 8. |
| Dataset Splits | Yes | Let Df = {(xf j , yf j )}q j=1 Dtr represent a forget set. The goal of machine unlearning is to remove the influence of Df from the original model, resulting in an unlearned model θu (i.e., θu = U(θo)) where U represents a machine unlearning algorithm. The unlearning algorithm may have access to other inputs (e.g., the validation set Dval) depending on the problem settings. Let Dr be the retain set, the subset of the training data excluding the forget set, i.e., Dr = Dtr \ Df. ... The auditing set Dθu is divided into the training Dtr θu and the validation Dval θu sets. ... For CIFAR-10 and CIFAR-100, the forget set consists of 10% of the entire training set, while the ratio is reduced to 5% for SVHN. In all experiments, the attacker s optimization problem is formulated as a binary classification task, where a linear support vector machine (SVM) is used to distinguish between forget and test instances. ... all of which are estimated on the auditing set with 10-fold cross Carlini et al. (2022). |
| Hardware Specification | Yes | We run all the experiments using Py Torch 1.12 on NVIDIA A5000 GPUs and AMD EPYC 7513 32-Core Processor. |
| Software Dependencies | Yes | We run all the experiments using Py Torch 1.12 on NVIDIA A5000 GPUs and AMD EPYC 7513 32-Core Processor. ... In practice, we describe the optimization problem Equation 7 using cvxpy (Diamond & Boyd, 2016). Then, we employ an off-the-shelf package called cvxpylayers (Agrawal et al., 2019b) to automatically derive the KKT conditions and compute the gradient θa/ Dθu. ... Due to the linear KKT condition in the decision variables, we leverage a specialized solver called qpth (Amos & Kolter, 2017) instead of using a more generic convex programming solver like cvxpylayer (Agrawal et al., 2019a). |
| Experiment Setup | Yes | For all methods, we use the SGD optimizer with a weight decay of 5e-4 and a momentum of 0.9. Other hyper-parameters are selected through the validation set. Specifically, we create a new auditing set. For each unlearning method, we select the hyper-parameters that maximize the difference between the validation accuracy and the MIA accuracy on this new auditing set. This approach ensures that the model both generalizes well to unseen data (high validation accuracy) and is less vulnerable to the attacks (low MIA accuracy). The hyperparameters are listed in Table 9 in the Appendix. |