Ctrl-U: Robust Conditional Image Generation via Uncertainty-aware Reward Modeling
Authors: Guiyu Zhang, Huan-ang Gao, Zijian Jiang, Hao Zhao, Zhedong Zheng
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments validate the effectiveness of our methodology in improving the controllability and generation quality, as well as its scalability across diverse conditional scenarios, including segmentation mask, edge, and depth conditions. Codes are publicly available at https://grenoble-zhang.github.io/Ctrl-U. |
| Researcher Affiliation | Academia | Guiyu Zhang 1,2 Huan-ang Gao 2 Zijian Jiang2 Hao Zhao 2 Zhedong Zheng 1 1 FST and ICI, University of Macau 2 AIR, Tsinghua University |
| Pseudocode | No | The paper describes the method in Section 3 and provides a pipeline overview in Figure 2, but there are no explicitly labeled pseudocode or algorithm blocks with structured steps. |
| Open Source Code | Yes | Codes are publicly available at https://grenoble-zhang.github.io/Ctrl-U. |
| Open Datasets | Yes | Our experiments are conducted using three datasets: ADE20K (Zhou et al., 2017; 2019), COCOStuff (Caesar et al., 2018) and Multi Gen-20M dataset (Qin et al., 2023), adhering to the dataset construction principles of Controlnet++ (Li et al., 2024a). |
| Dataset Splits | Yes | We use the ADE20K dataset for segmentation masks, which includes 20,210 images in the training set and 2,000 images in the validation set. [...] Similarly, COCO-Stuff provides segmentation annotations, with 118,287 images in the training set and 5,000 in the validation set. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. It only discusses general training settings. |
| Software Dependencies | No | The paper mentions using Adam as the optimizer and refers to the pre-trained Control Net model, but does not provide specific version numbers for any key software components or libraries (e.g., Python, PyTorch, TensorFlow). |
| Experiment Setup | Yes | In our experiments, we first fine-tune the pre-trained Control Net model to convergence, using Adam as the optimizer with a learning rate of 1e-5, weight decay of 1e-2, and momentum of 0.9. Then, we use the same optimization settings to perform 10k iterations of uncertainty-aware reward fine-tuning. |