JPEG Inspired Deep Learning
Authors: Ahmed Hussien Salamah, Kaixiang Zheng, Yiwen Liu, EN-HUI YANG
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To validate the effectiveness of JPEG-DL, we conducted extensive experiments for image classification on six datasets including four fine-grained classification datasets (Wah et al., 2011; Khosla et al., 2011; Nilsback & Zisserman, 2008; Parkhi et al., 2012), CIFAR-100 (Krizhevsky et al., 2009) and Image Net (Deng et al., 2009). Results show that JPEG-DL significantly and consistently outperforms the standard DL across various DNN architectures, with a negligible increase in model complexity. |
| Researcher Affiliation | Academia | Ahmed H. Salamah , Kaixiang Zheng , Yiwen Liu & En-Hui Yang Department of Electrical and Computer Engineering, University of Waterloo EMAIL |
| Pseudocode | No | The paper describes mathematical formulations and processes (like the JPEG pipeline and differentiable soft quantizer) in paragraph form and with equations, but does not present any explicit pseudocode blocks labeled 'Algorithm' or 'Pseudocode'. |
| Open Source Code | Yes | Our code is available on https://github.com/AhmedHussKhalifa/JPEG-Inspired-DL.git. |
| Open Datasets | Yes | To validate the effectiveness of JPEG-DL, we conducted extensive experiments for image classification on six datasets including four fine-grained classification datasets (Wah et al., 2011; Khosla et al., 2011; Nilsback & Zisserman, 2008; Parkhi et al., 2012), CIFAR-100 (Krizhevsky et al., 2009) and Image Net (Deng et al., 2009). |
| Dataset Splits | Yes | For CIFAR-100, we evaluate our proposed method... For fine-grained tasks... We evaluate our method on four datasets: CUB-200-2011 (Wah et al., 2011), Stanford Dogs (Khosla et al., 2011), Flowers (Nilsback & Zisserman, 2008), and Pets (Parkhi et al., 2012)... For Image Net-1K. For all experiments on this dataset, we utilize the standard training recipes shown by Paszke et al. (2019) without any modifications. |
| Hardware Specification | Yes | Evaluation is conducted on a single NVIDIA RTX A5000 GPU. |
| Software Dependencies | No | The paper mentions using specific optimizers like SGD and Adam, and cites PyTorch, but does not provide specific version numbers for software libraries or dependencies used in their experimental setup. |
| Experiment Setup | Yes | For CIFAR-100, we set the learning rate to 0.003 across all tested models. For the fine-grained datasets, we set the JPEG learning rate to 0.005. Across these datasets, we fix αm = 5 for all 1 m 2M, and set L = 2b 1 in (3), where b is a tunable hyperparameter set to 8. ... We define the Gradient Scaling Constants ℏm = αmq2 m, 1 m 2M, which allows us to control the magnitude of gradients w.r.t Q. Specifically, we fix ℏm = 0.7 for all 1 m 2M. ... For this dataset, b is equal to 11 across all tested models. ... For CIFAR100, we deploy a stochastic gradient descent (SGD) optimizer with a momentum of 0.9, a weight decay of 0.0005, and a batch size of 64. We initialize the learning rate as 0.05, and decay it by 0.1 every 30 epochs after the first 150 epochs until the last 240 epoch. ... For fine-grained classification tasks, all networks are trained from scratch and optimized by SGD with a momentum of 0.9, weight decay of 0.0001, and an initial learning rate of 0.1,. The learning rate is divided by 10 after epochs 100 and 150 for all datasets, and the total epochs are 200. We set batch size 32 for these fine-grained classification tasks. |