Contrastive Unlearning: A Contrastive Approach to Machine Unlearning
Authors: Hong kyu Lee, Qiuchen Zhang, Carl Yang, Jian Lou, Li Xiong
IJCAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on a variety of datasets and models on both class unlearning and sample unlearning showed that contrastive unlearning achieves the best unlearning effects and efficiency with the lowest performance loss compared with the state-of-the-art algorithms. |
| Researcher Affiliation | Academia | Hong kyu Lee , Qiuchen Zhang , Carl Yang , Jian Lou and Li Xiong Emory University EMAIL |
| Pseudocode | Yes | The full algorithm is in Appendix B. |
| Open Source Code | Yes | Our main code is available on github.com/Emory AIMS/Contrastive-Unlearning |
| Open Datasets | Yes | We use three benchmark datasets: CIFAR-10, SVHN, and Mini-Imagenet [Cao, 2022], and employ Res Net (RN)-18, 34, 50, and 101 models [He et al., 2016] and Vi T-small [Dosovitskiy et al., 2021]. |
| Dataset Splits | Yes | For class unlearning, we remove all samples belonging to class 5 by default. For sample unlearning, we remove randomly selected 500 samples by default. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for key software components or libraries used. |
| Experiment Setup | Yes | L = εULLUL + εCELCE (F (Xr) , Y r) where Xr and Y r are the sampled batches of remaining samples and their corresponding labels. ςCE and ςUL are hyperparameters to determine influence of the two loss terms. The full algorithm is in Appendix B. Termination Condition. Pinpointing the right moment to terminate the unlearning process is crucial, as terminating too early or too late will lead to insufficient unlearning or poor model utility. None of existing works explicitly discuss the termination condition. We design explicit termination conditions for both class and sample unlearning based on our unlearning goals. We assume a small dataset Deval is available for determining the termination condition. We evaluate the conditions every unlearning round. For class unlearning, recall our problem definition in 3.1 and the goal in equation 1. We can set Deval = Du ts, the test data of the unlearning class. Ideally, we want F to have close to 0 accuracy for the unlearning class. However, this can be too strict for termination. We loosen the condition and terminate the algorithm when the accuracy of F on the unlearning class falls below a threshold. We set the threshold to be 1/C where C is the total number of classes in the training data and 1/C corresponds to the accuracy of a random guess, which suggests knowledge about the unlearning class is sufficiently removed from the model. ... For sample unlearning, recall our problem definition of 3.2 and the goal in equation 3. Specifically, we set Deval = {Du eval, Dts eval} where Du eval Du tr and Dts eval Dts. The algorithm terminates when the accuracy of F on Du eval drops below the accuracy on Dts eval. ... For class unlearning, we remove all samples belonging to class 5 by default. For sample unlearning, we remove randomly selected 500 samples by default. ... We repeat sample unlearning with a random seed for five times and report the average and standard deviation of the results. |