Maximising the Utility of Validation Sets for Imbalanced Noisy-label Meta-learning
Authors: Hoang Anh Dung, Cuong C. Nguyen, Vasileios Belagiannis, Thanh-Toan Do, Gustavo Carneiro
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The proposed method shows state-of-the-art (SOTA) results compared to previous meta-learning and noisy-label learning approaches on several noisy-label learning benchmarks. 5 Experiments The proposed method INOLML is evaluated on several datasets, including CIFAR10, CIFAR100, mini Web Vision and Controlled Noisy Web Labels (CNWL). |
| Researcher Affiliation | Academia | Dung Anh Hoang EMAIL Department of Data Science and AI Monash University Cuong Nguyen EMAIL Centre for Vision, Speech and Signal Processing University of Surrey Vasileios Belagiannis EMAIL Friedrich-Alexander-Universität Erlangen-Nürnberg Thanh-Toan Do EMAIL Department of Data Science and AI Monash University G. Carneiro EMAIL Centre for Vision, Speech and Signal Processing University of Surrey |
| Pseudocode | Yes | Algorithm 1 Training procedure of the proposed INOLML. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper. |
| Open Datasets | Yes | The proposed method INOLML is evaluated on several datasets, including CIFAR10, CIFAR100, mini Web Vision and Controlled Noisy Web Labels (CNWL). Both CIFAR10 and CIFAR100 datasets (Krizhevsky & Hinton, 2009) contain 50k and 10k images used for training and testing, respectively. Web Vision (Li et al., 2017) is a dataset of 2.4 million images crawled from Google and Flickr... |
| Dataset Splits | Yes | Both CIFAR10 and CIFAR100 datasets (Krizhevsky & Hinton, 2009) contain 50k and 10k images used for training and testing, respectively. Red mini-Image Net dataset consisting of 50k training images from 100 classes for training and 5k images for testing. For instance, we need around K = 200 candidate samples per class for D(v) before selecting M = 10 samples per class for the validation set D(v) (e.g., for CIFAR100, we need 20,000 samples for D(v)) |
| Hardware Specification | Yes | The experiment above was conducted on a single NVIDIA V100 GPU. For the Web Vision experiment, we use p = 4, k = 8 with 4 NVIDIA V100 GPU and batches of size 16. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | For CIFAR, we train Wide Resnet28-10 with 100k iterations and a batch size of 100. For mini-Web Vision, we follow FSR (Zhang & Pfister, 2021) and train a single Resnet50 network with 1 million iterations and a batch size of 16. All experiments use N = 200, K = 50, κ = 0.9. |