ConMix: Contrastive Mixup at Representation Level for Long-tailed Deep Clustering
Authors: Zhixin Li, Yuheng Jia
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on widely used benchmark datasets with different imbalance ratios, suggesting it outperforms many state-of-the-art deep clustering approaches. The code is available at https://github.com/LZX-001/Con Mix. 5 EXPERIMENTS 5.1 EXPERIMENTAL SETUP 5.2 MAIN RESULTS 5.3 ABLATION STUDY 5.4 CLUSTERING IN MORE LONG-TAILED SCENARIOS 5.5 CLUSTERING ON THE LARGE-SCALE DATASET 5.6 CONMIX BENEFITS REPRESENTATION LEARNING 5.7 CONMIX WORKS ON UNSEEN BALANCED DATASETS |
| Researcher Affiliation | Academia | Zhixin Li1, Yuheng Jia1,2 1 School of Computer Science and Engineering, Southeast University, Nanjing 210096, China 2 Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China EMAIL |
| Pseudocode | No | The paper describes the method and its components in prose, including equations, but does not present a clearly labeled 'Pseudocode' or 'Algorithm' block with structured steps. |
| Open Source Code | Yes | We evaluate our method on widely used benchmark datasets with different imbalance ratios, suggesting it outperforms many state-of-the-art deep clustering approaches. The code is available at https://github.com/LZX-001/Con Mix. |
| Open Datasets | Yes | We conduct experiments on three benchmark datasets, including CIFAR-10 (Krizhevsky et al., 2009), CIFAR-20 (Krizhevsky et al., 2009) and STL-10 (Coates et al., 2011). ... Besides, we have also conducted experiments on CIFAR10 with an imbalance ratio of 100 and on Tiny Image Net (Le and Yang, 2015) with an imbalance ratio of 10... We also report clustering results on Image Net-LT (Liu et al., 2019) in Appendix F to demonstrate the effectiveness and generalization of Con Mix. |
| Dataset Splits | Yes | We manually generated a total of six long-tailed distributions using two imbalance ratios: 5 and 10. The imbalance ratio, which is the ratio of the maximum and minimum classes, controls the long-tailed degree of data distributions. We generate the sample number for each class following the setting in (Tang et al., 2020; Zhou et al., 2020; Cao et al., 2019), where the number of each class in long-tailed distributions exponentially decreases. We use the train dataset in CIFAR-10 and CIFAR-20, both train and test dataset in STL-10 (due to its small train dataset) for long-tailed deep clustering, following (Tao et al., 2020). ... In test set of CIFAR-10, each class has 1,000 samples, which are not present in the train set. |
| Hardware Specification | Yes | Our experiments are based on Pytorch and all models are trained on NVIDIA Ge Force RTX 4090 GPUs. It takes approximately five to seven hours on a single GPU to train a model under different long-tailed distributions in the Section 5.2, with the actual time depending on factors such as the size of the dataset. |
| Software Dependencies | No | Our experiments are based on Pytorch and all models are trained on NVIDIA Ge Force RTX 4090 GPUs. While PyTorch is mentioned, no specific version number is provided for it or any other software library. |
| Experiment Setup | Yes | We train all methods for 1000 epochs and report results of the last epoch for fair comparisons. We use a batch size of 512 for all methods unless otherwise specified. We hope to mixup semantically meaningful representation, thus we train our model only using SDCLR (Jiang et al., 2021) for the first 200 epochs. Then we train our model only with Con Mix. We set synthesized representation number M = 100 in all experiments... Following compared methods (Huang et al., 2023; Yu et al., 2023; Tao et al., 2020; Li et al., 2023), we use K-means for final representation clustering. Other specific training settings can be found in the Appendix A. ... We use Res Net-18 (He et al., 2016) as the backbone for all experiments... For Con Mix, we adopt the stochastic gradient descent (SGD) optimizer, whose learning rate is 0.5, weight decay is 0.0001 and momentum is 0.9. We adopt the cosine decay learning rate schedule to update the learning rate by step, with 10 epochs for learning rate warmup. The temperature τ in NT-Xent is 0.2. |