Optimized Tradeoffs for Private Prediction with Majority Ensembling
Authors: Shuli Jiang, Qiuyi Zhang, Gauri Joshi
TMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Lastly, we demonstrate the strong empirical effectiveness of our first-of-its-kind privacy-constrained utility optimization for ensembling labels for private prediction from private teachers in image classification. Notably, our Da RRM framework with an optimized γ exhibits substantial utility gains when compared against several baselines. Experiments. In downstream tasks, such as semi-supervised knowledge transfer for private image classification, we compare our Da RRM with an optimized γ to compute the private label majority from private teachers against PATE Papernot et al. (2018), which computes the private label majority from non-private teachers. |
| Researcher Affiliation | Collaboration | Shuli Jiang EMAIL Robotics Institute, Carnegie Mellon University Qiuyi (Richard) Zhang EMAIL Google Deep Mind Gauri Joshi EMAIL Electrical and Computer Engineering, Carnegie Mellon University |
| Pseudocode | Yes | Algorithm 1 Da RRM( ): Data-dependent Randomized Response Majority |
| Open Source Code | Yes | All code for the experiments can be found at https://anonymous.4open.science/r/Optimized Private Majority-CF50 |
| Open Datasets | Yes | We use samples from two randomly chosen classes class 5 and 8 from the MNIST and Fashion-MNIST datasets to form our training and testing datasets. |
| Dataset Splits | Yes | Our MNIST has a total of 11272 training samples and 1866 testing samples; our Fashion-MNIST has 10000 training samples and 2000 testing samples. ... We train K = 11 teachers on equally divided subsets of the training datasets. |
| Hardware Specification | No | The paper mentions 'In practice, we observe with the Gurobi optimizer, one can optimize γ for K 41 on a laptop if δ > 0.' This refers to a practical limitation for optimization, not the specific hardware used for running the primary experiments or model training, and no specific model is mentioned. |
| Software Dependencies | No | The paper mentions 'Gurobi solver' and 'DP-SGD Abadi et al. (2016)' but does not provide specific version numbers for these software components or any other libraries or programming languages used. |
| Experiment Setup | Yes | Da RRM Setup: The Gaussian noise in DP-SGD has zero mean and std. σdpsgd = 12; the gradient norm clipping threshold is C = 1. ... We set the privacy allowance m = 35 ... We train K = 11 teachers ... for 5 epochs. ... We pick Q {20, 50, 100}. |