Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement
Authors: Hyeonjin Kim, Jaejun Yoo
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments with representative generative model architectures across various datasets, including Style GAN2, Style GAN3, and Denoising Diffusion Probabilistic Model (DDPM) on CIFAR10, Celeb A-HQ, FFHQ, and LSUN-Church. The results demonstrate that our method enhances the fine-tuning process, leading to both faster convergence and improved solutions for the compressed model without additional training cost. |
| Researcher Affiliation | Academia | Hyeonjin Kim and Jaejun Yoo* Graduate School of Artificial Intelligence Ulsan National Institute of Science and Technology (UNIST) EMAIL |
| Pseudocode | No | The paper describes the method and its mathematical formulation (Wscaled = UΣscaled V T) but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper mentions implementing their method based on existing open-source implementations (e.g., "official Style GAN2 implementation", "DCPGAN implementation", "Diff-Prune"), but there is no explicit statement or link indicating that the authors' own code for this work is publicly available. |
| Open Datasets | Yes | We conduct extensive experiments with representative generative model architectures across various datasets, including Style GAN2, Style GAN3, and Denoising Diffusion Probabilistic Model (DDPM) on CIFAR10, Celeb A-HQ, FFHQ, and LSUN-Church. The paper provides citations for these datasets: "CIFAR10 (Alex 2009)", "Celeb A-HQ (Liu et al. 2015)", "FFHQ (Karras, Laine, and Aila 2019)", and "LSUN Church (Yu et al. 2015)". |
| Dataset Splits | No | The paper describes how samples are used for evaluating metrics (e.g., "To calculate FID, we use all real samples from each dataset and 50K fake samples"), but it does not provide explicit training/validation/test splits for the datasets used to train the models. For DDPM, it refers to "following the experimental setup of Diff-Prune" without specifying splits in the main text. |
| Hardware Specification | No | The paper does not provide any specific details regarding the hardware (e.g., GPU models, CPU types, memory specifications) used for conducting the experiments. |
| Software Dependencies | No | The paper mentions using specific implementations like the "official Style GAN2 implementation" and building upon "DCPGAN implementation" and "Diff-Prune", but it does not specify version numbers for any software libraries, frameworks, or programming languages used (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | No | The paper refers to existing experimental setups (e.g., "following the experimental setup of DCP-GAN" and "following the experimental setup of Diff-Prune") and mentions some high-level training parameters like "train Style GAN3 on the FFHQ dataset until the discriminator see 10 million images" or "Train Steps" for DDPM. However, it explicitly states, "We provide more detailed implementation in the supplementary material," indicating that comprehensive experimental setup details (like learning rates, batch sizes, optimizers, etc.) are not present in the main text. |