Breaking the Reclustering Barrier in Centroid-based Deep Clustering
Authors: Lukas Miklautz, Timo Klein, Kevin Sidak, Collin Leiber, Thomas Lang, Andrii Shkabrii, Sebastian Tschiatschek, Claudia Plant
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We integrate BRB into DEC, IDEC, and DCN, presenting experiments that demonstrate the performance improvements resulting from this integration for a wide range of datasets. The first part of our analysis describes the datasets and setups used in our experiments. We then proceed to benchmark BRB against unmodified versions of DEC, IDEC, and DCN in three scenarios: With and without pre-training and with a contrastive auxiliary task. Then, we dig deeper into the underlying mechanisms of BRB and examine how it is able to break the reclustering barrier. Overall, we find that BRB improves performance in 88.10 % of all runs while incurring a minimal runtime overhead of approximately 1.1 %. |
| Researcher Affiliation | Academia | 1 Faculty of Computer Science, University of Vienna, Vienna, Austria 2 Uni Vie Doctoral School Computer Science, University of Vienna, Vienna, Austria 3 Department of Computer Science, Aalto University, Espoo, Finland 4 Department of Computer Science, University of Helsinki, Helsinki, Finland 5 ds:Uni Vie, Vienna, Austria |
| Pseudocode | Yes | REPRODUCIBILITY Our code and models are publicly available on github at https://github.com/Probabilistic-and-Interactive-ML/ breaking-the-reclustering-barrier. Additionally, we provide pseudocode of our algorithm in Section A and all experiment hyperparameters together with implementation details in Appendix Section E. ... Algorithm 1 Py Torch-style pseudo-code of BRB |
| Open Source Code | Yes | We release our code and pre-trained models at https://github.com/Probabilistic-and-Interactive-ML/ breaking-the-reclustering-barrier. |
| Open Datasets | Yes | We evaluate BRB on eight common deep clustering datasets, divided into two groups, each with a distinct setup. Dataset and preprocessing details are provided in Appendix C. The first group (MNIST (Le Cun et al., 1998), KMNIST (Clanuwat et al., 2018), FMNIST (Xiao et al., 2017), USPS (Hull, 1994), OPTDIGITS (Alpaydin & Kaynak, 1998), and GTSRB (Stallkamp et al., 2012)) uses a feed-forward autoencoder with reconstruction as an auxiliary task, a learning rate of 0.001, and fixed BRB hyperparameters α = 0.8 and T = 20 across all algorithms and setups. The second group (CIFAR10 and CIFAR100-20 (Krizhevsky et al., 2009)), consisting of more challenging color image datasets, uses Sim CLR (Chen et al., 2020) as an auxiliary task, a Res Net18 (He et al., 2016) encoder, and self-labeling (Gansbeke et al., 2020) after clustering as in (Qian, 2023). |
| Dataset Splits | Yes | We evaluate BRB on eight common deep clustering datasets, divided into two groups, each with a distinct setup. Dataset and preprocessing details are provided in Appendix C. The first group (MNIST (Le Cun et al., 1998), KMNIST (Clanuwat et al., 2018), FMNIST (Xiao et al., 2017), USPS (Hull, 1994), OPTDIGITS (Alpaydin & Kaynak, 1998), and GTSRB (Stallkamp et al., 2012))... The second group (CIFAR10 and CIFAR100-20 (Krizhevsky et al., 2009))... reporting the best performance after self-labeling on the respective test sets of CIFAR10 and CIFAR100-20. |
| Hardware Specification | Yes | We primarily utilized internal servers to generate our results with grayscale datasets. For this paper, we had access to 2 Intel Xeon Silver 4214R CPU @ 2.40GHz server with 48 cores and 2 Nvidia A100 GPUs with 40GB of VRAM each, 2 Intel Xeon Gold 6326 CPU @ 2.90GHz server with 64 cores and 1 NVIDIA A100 GPU with 80GB of VRAM. These machines were also used to run preliminary experiments and to identify broad working ranges for the most important hyperparameters. For the experiments with color datasets and contrastive learning, we had access to 1000h compute hours per month on a SLURM supercomputing cluster. A node in this supercomputer uses an AMD EPYC 7452 processor with 128 cores and 1 NVIDIA A100 GPU with 80GB of VRAM. |
| Software Dependencies | No | Explanation: The paper mentions Py Torch (Paszke et al., 2019), Num Py (Harris et al., 2020), Matplotlib (Hunter, 2007), and Clust Py (Leiber et al., 2023) as tools used, along with their respective citations, but does not specify exact version numbers for these software dependencies (e.g., PyTorch 1.9 or Python 3.8). |
| Experiment Setup | Yes | The first group (MNIST (Le Cun et al., 1998), KMNIST (Clanuwat et al., 2018), FMNIST (Xiao et al., 2017), USPS (Hull, 1994), OPTDIGITS (Alpaydin & Kaynak, 1998), and GTSRB (Stallkamp et al., 2012)) uses a feed-forward autoencoder with reconstruction as an auxiliary task, a learning rate of 0.001, and fixed BRB hyperparameters α = 0.8 and T = 20 across all algorithms and setups. The second group (CIFAR10 and CIFAR100-20 (Krizhevsky et al., 2009)), consisting of more challenging color image datasets, uses Sim CLR (Chen et al., 2020) as an auxiliary task, a Res Net18 (He et al., 2016) encoder, and self-labeling (Gansbeke et al., 2020) after clustering as in (Qian, 2023). See Appendix E for full experimental details. |