Fixed Point Quantization of Deep Convolutional Networks
Authors: Darryl Lin, Sachin Talathi, Sreekanth Annapureddy
ICML 2016 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer > 20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark. |
| Researcher Affiliation | Industry | Darryl D. Lin EMAIL Qualcomm Research, San Diego, CA 92121, USA Sachin S. Talathi EMAIL Qualcomm Research, San Diego, CA 92121, USA V. Sreekanth Annapureddy EMAIL Netra Dyne Inc., San Diego, CA 92121, USA |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link indicating that the source code for the described methodology is publicly available. |
| Open Datasets | Yes | Our experiments show that in comparison to equal bit-width settings, the fixed point DCNs with optimized bit width allocation offer > 20% reduction in the model size without any loss in accuracy on CIFAR-10 benchmark. We also demonstrate that fine-tuning can further enhance the accuracy of fixed point DCNs beyond that of the original floating point model. In doing so, we report a new state-of-the-art fixed point performance of 6.78% error-rate on CIFAR-10 benchmark. ... Here we carry out a similar exercise for an Alex Netlike DCN that is trained on Image Net-1000. |
| Dataset Splits | No | The paper mentions using CIFAR-10 and ImageNet benchmarks, which typically have predefined splits, but it does not explicitly state the specific training, validation, and test split percentages or sample counts used for reproduction. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not list any specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow, CUDA versions) that would be needed to replicate the experiments. |
| Experiment Setup | Yes | Table 2. Parameters per layer in our CIFAR-10 network ... Table 4. Parameters per layer in our Alex Net implementation ... Table 7 contains the classification error rate (in %) for the CIFAR-10 network after fine-tuning the model for 30 epochs. We experiment with different weight and activation bit-width combinations, ranging from floating point to 4-bit, 8-bit, and 16-bit fixed point. |