DIVINE: Diverse-Inconspicuous Feature Learning to Mitigate Abridge Learning
Authors: Saheb Chhabra, Kartik Thakral, Surbhi Mittal, Mayank Vatsa, Richa Singh
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments on multiple datasets, including MNIST, CIFAR-10, CIFAR-100, Tiny Image Net, and their corrupted and perturbed variants (CIFAR-10-C/P, CIFAR-100-C/P, Tiny Image Net-C/P), we demonstrate that DIVINE significantly improves model robustness and generalization. On perturbation benchmarks, DIVINE achieves mean Flip Rates (m FR) of 5.36%, 3.10%, and 21.85% on CIFAR-10-P, CIFAR-100-P, and Tiny Image Net-P respectively, compared to 6.53%, 11.75%, and 31.90% for standard training methods exhibiting Abridge Learning. |
| Researcher Affiliation | Academia | Saheb Chhabra EMAIL Department of Computer Science IIIT-Delhi, New Delhi, India 110020 Kartik Thakral EMAIL Department of Computer Science and Engineering IIT Jodhpur, Rajasthan, India 342037 Surbhi Mittal EMAIL Department of Computer Science and Engineering IIT Jodhpur, Rajasthan, India 342037 Mayank Vatsa EMAIL Department of Computer Science and Engineering IIT Jodhpur, Rajasthan, India 342037 Richa Singh EMAIL Department of Computer Science and Engineering IIT Jodhpur, Rajasthan, India 342037 |
| Pseudocode | Yes | As illustrated in Figure 2, the proposed DIVINE algorithm comprises a two-stage learning strategy designed explicitly to counteract the effects of Abridge Learning. In the first stage, DIVINE systematically identifies dominant input features using dominance feature map, which quantifies the relative importance of each input feature for classification. Once these dominant features are identified, the algorithm suppresses them, compelling the model to discover previously overlooked yet informative inconspicuous features. Subsequently, in the second stage, a unified model leverages both dominant and newly identified inconspicuous features to enhance the overall generalization performance. |
| Open Source Code | Yes | The source code is available at https://github.com/Saheb-Chhabra/ID-Learning/. |
| Open Datasets | Yes | Extensive experiments on benchmark datasets such as CIFAR10 (Krizhevsky & Hinton, 2009), MNIST (Le Cun et al., 1998), CIFAR10-C (Hendrycks & Dietterich, 2019b), and CIFAR100-C (Hendrycks & Dietterich, 2019b) highlight DIVINE s effectiveness and confirm its broad applicability across multiple machine learning tasks. |
| Dataset Splits | Yes | MNIST (Le Cun et al., 1998) consists of 60,000 training images and 10,000 testing images, of handwritten digits from 10 different classes (0-9). Each image is a grayscale image of 28 28 resolution. The standard pre-defined protocol is used for evaluation. CIFAR10 (Krizhevsky & Hinton, 2009) contains 60,000 32 32 color images of 10 different classes with 50,000 images in training set and 10,000 images in testing set. The standard pre-defined protocol is used for evaluation. CIFAR100 (Krizhevsky & Hinton, 2009) contains 60,000 32 32 color images across 100 finegrained classes grouped into 20 superclasses with 50,000 images in training set and 10,000 images in testing set. The standard pre-defined protocol is used for evaluation. |
| Hardware Specification | Yes | The model trainings are performed on a DGX station with Intel Xeon CPU, 256 GB RAM, and four 32 GB Nvidia V100 GPU cards. |
| Software Dependencies | Yes | Code is implemented in Tensorflow 2.3.1. |
| Experiment Setup | Yes | Models are trained on the original and feature-suppressed datasets with a learning rate of 0.0001. For the CIFAR10 and CIFAR100 datasets, models are trained using Adam optimizer with a batch size of 32 whereas, models are trained using RMSProp optimizer with a batch size of 64 for the Tiny Image Net dataset. For the MNIST dataset, the models are trained for 10 epochs and the features are suppressed with p = 3%. The models on the CIFAR10 and CIFAR100 datasets are trained for 20 epochs and p = 3% is used. The unified models are trained using Adam optimizer with a learning rate of 0.0001 for 20 epochs and batch size of 32. |