Early Stopping for Deep Image Prior
Authors: Hengkang Wang, Taihui Li, Zhong Zhuang, Tiancong Chen, Hengyue Liang, Ju Sun
TMLR 2023 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this paper, we set out to break this practicality barrier of DIP and propose an effective ES strategy that consistently detects near-peak performance in various computational imaging tasks and DIP variants. Simply based on the running variance of DIP intermediate reconstructions, our ES method not only outpaces the existing ones which only work in very narrow regimes, but also remains effective when combined with methods that try to mitigate overfitting. The code to reproduce our experimental results is available at https://github.com/sun-umn/Early_Stopping_for_DIP. |
| Researcher Affiliation | Academia | Hengkang Wang EMAIL University of Minnesota, Twin Cities Taihui Li EMAIL University of Minnesota, Twin Cities Zhong Zhuang EMAIL University of Minnesota, Twin Cities Tiancong Chen EMAIL University of Minnesota, Twin Cities Hengyue Liang EMAIL University of Minnesota, Twin Cities Ju Sun EMAIL University of Minnesota, Twin Cities |
| Pseudocode | Yes | Algorithm 1 DIP with ES WMV Algorithm 2 DIP with ES EMV |
| Open Source Code | Yes | The code to reproduce our experimental results is available at https://github.com/sun-umn/Early_Stopping_for_DIP. |
| Open Datasets | Yes | We randomly sample 1024 images from the RGB track of the NTIRE 2020 Real Image Denoising Challenge (Abdelhamed et al., 2020), and perform DIP-based image denoising. We take the classical 9image dataset (Dabov et al., 2008) We test both ES-WMV and SB on their modified DIP model in (Shi et al., 2022), based on the two datasets they test: the classic 9image dataset (Dabov et al., 2008) and the CBSD68 dataset (Martin et al., 2001). We test our ES-WMV for DIP and a state-of-the-art zero-shot method based on pre-trained diffusion model DDNM+ Wang et al. (2022) on the standard super-resolution dataset Set14 Zeyde et al. (2012) We visualize the performance on two random cases (C1: 1001339 and C2: 1000190 sampled from Darestani & Heckel (2021), part of the fast MRI datatset (Zbontar et al., 2018)) We systematically test our ES-WMV and VAL on the entire standard Levin dataset for both low-level and high-level cases. We conduct experiments on the Kodak dataset3 and prepare it following the pipeline in Li et al. (2023a). |
| Dataset Splits | No | The paper mentions evaluating on several datasets, often describing how data is generated or corrupted (e.g., 'for each noise type, generate two noise levels', 'Mask m is generated according to an i.i.d. Bernoulli model with a rate of 50%', '8-fold undersampling'). However, it does not explicitly specify how these datasets are formally divided into training, validation, or test sets for model development and evaluation in the traditional supervised learning sense. DIP models typically operate on single instances rather than being trained on large datasets with predefined splits. |
| Hardware Specification | Yes | Table 3: Wall-clock time (secs) of DIP and three ES methods per epoch on NVIDIA Tesla K40 GPU: mean and (std). |
| Software Dependencies | No | The paper provides URLs for external codebases used or compared against, but it does not specify the version numbers for general software dependencies like Python, PyTorch, or CUDA that were used to run their experiments. |
| Experiment Setup | Yes | Our default setup for all experiments is as follows. Our DIP model is the original from Ulyanov et al. (2018); the optimizer is ADAM with a learning rate 0.01. For all other models, we use their default architectures, optimizers, and hyperparameters. For ES-WMV, the default window size W = 100, and the patience number P = 1000. |