PrivShap: A Finer-granularity Network Linearization Method for Private Inference
Authors: Xiangrui Xu, Zhenzhen Wang, Rui Ning, Chunsheng Xin, Hongyi Wu
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our extensive experiments show that we achieve better efficiency and accuracy than the state-of-the-art across diverse model architectures, activation functions, and datasets. Specifically, we only need 2.5 fewer Re LU operations to achieve a similar inference accuracy and gains up to 8.13% increase on inference accuracy with similar Re LU budgets. |
| Researcher Affiliation | Academia | Xiangrui Xu EMAIL Department of Computer Science Old Dominion University Zhenzhen Wang EMAIL Department of Biomedical Engineering Johns Hopkins University Rui Ning EMAIL Department of Computer Science Old Dominion University Chunsheng Xin EMAIL Department of Computer Science Iowa State University Hongyi Wu EMAIL Department of Electrical and Computer Engineering University of Arizona |
| Pseudocode | Yes | Algorithm 1 Layer-wise Re LU importance Estimation |
| Open Source Code | No | The paper does not provide explicit statements about releasing source code for their methodology, nor does it provide a direct link to a code repository. |
| Open Datasets | Yes | We apply Priv Shap to models such as Res Net18, Res Net34 with Si LU activation function, and Wide Res Net-22 in three benchmark datasets: CIFAR-10, CIFAR-100 and Tiny-Image Net. |
| Dataset Splits | Yes | CIFAR-10 contains 10 classes with 5,000 training images and 1,000 test images per class, whereas CIFAR-100 includes 100 classes, with 500 training images and 100 test images per class. Tiny-Image Net has a larger image resolution of 64 64 and consists of 200 classes, each with 500 training and 50 validation images. |
| Hardware Specification | Yes | The experiments are conduct on two servers with an AMD EPYC 7413 24-core Processor 64GB RAM, under WAN (1 GBps, 1 ms latency) and LAN (300MBps, 5 ms latency). |
| Software Dependencies | No | The paper mentions optimizers like SGD and Adam, and the SEAL library, but does not provide specific version numbers for these software components or other key dependencies like programming languages or deep learning frameworks. |
| Experiment Setup | Yes | We first train the all-Re LU (AR) models on CIFAR-10, CIFAR-100, and Tiny-Image Net datasets using the SGD optimizer with an initial learning rate of 0.1, momentum of 0.9, and weight decay of 0.0005. The learning rate decays by a factor of 0.1 at 80 and 120 epochs for CIFAR-10/100, using a batch size of 256. For Tiny-Image Net, we train for 200 epochs with the same hyperparameters. After this, we apply gradientprojection pruning for partial-Re LU (PR) models with a trimming threshold of 5% and an initial learning rate of 0.001 using the Adam optimizer. |