Towards hyperparameter-free optimization with differential privacy
Authors: Ruixuan Liu, Zhiqi Bu
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show the strong performance and high efficiency of our method empirically. ... 5 EXPERIMENTS ... Table 2: Performance comparison of Hy Free DP with other baselines. |
| Researcher Affiliation | Collaboration | Ruixuan Liu Emory University EMAIL Zhiqi Bu Amazon EMAIL |
| Pseudocode | Yes | Algorithm 1 Hyperparameter-free Optimization with Differential Privacy |
| Open Source Code | No | We use auto DP library2 to compute σl based on σg in (10). https://github.com/yuxiangw/autodp ... 4https://github.com/lxuechen/private-transformers ... The paper discusses the use of third-party open-source libraries but does not provide concrete access to its own source code for the methodology described in this paper. |
| Open Datasets | Yes | we conduct experiments on both computer vision tasks and natural language tasks, across different model architectures ... on CIFAR10, CIFAR100, SVHN, GTSRB and Food101 ... on E2E dataset ... on Pub Med dataset ... use the dataset provided by Yu et al. (2022; 2023) |
| Dataset Splits | No | For LLa Ma2-7B, we sample 4,000 records from Pub Med and train for 3 epochs on a single A100 (80GB). ... we use the privacy budget ϵ = {1, 3, 8} with δ N 1.1 for training dataset with N samples. ... The paper uses well-known datasets but does not explicitly provide specific details about training/validation/test splits (e.g., percentages, sample counts, or citations to predefined splits) for reproduction. |
| Hardware Specification | Yes | For LLa Ma2-7B, we sample 4,000 records from Pub Med and train for 3 epochs on a single A100 (80GB). For smaller datasets and models, we use previous setups on Titan RTX (24GB). |
| Software Dependencies | No | we fix the momentum coefficients and weight decay (β1, β2, weight decay) = (0.9, 0.999, 0.01), which is the default in Pytorch Adam W. ... We use auto DP library2 to compute σl based on σg in (10). ... Solve( ) such as the bi-section in scipy library. The paper mentions software components like Pytorch Adam W, auto DP library, and scipy library, but does not provide specific version numbers for them. |
| Experiment Setup | Yes | we fix the momentum coefficients and weight decay (β1, β2, weight decay) = (0.9, 0.999, 0.01), which is the default in Pytorch Adam W. ... INPUT: initial η=1e-4, initial Rl = 1 ... if t%K == 0 (e.g. K = 10) then ... For LLa Ma2-7B, we sample 4,000 records from Pub Med and train for 3 epochs... We use the privacy budget ϵ = {1, 3, 8} with δ N 1.1... |