When resampling/reweighting improves feature learning in imbalanced classification? A toy-model study
Authors: Tomoyuki Obuchi, Toshiyuki Tanaka
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In sec. 4, numerical experiments are conducted to verify the replica results. Our theoretical framework is built upon a method that assumes N . Thus, if our numerical results, computed with sufficiently large N, align well with the theoretical findings, this would support the validity of our analysis. For this purpose, we only examined the case with CElo in this section since the CE loss is convex and thus the numerical optimization is relatively easy; such a good property is absent in the zero-one loss. The standard interior-point method was used for the optimization. In the following results, we conducted simulations with N = 400 and took a sample average over 100 different realizations of the dataset; the error bar is the standard error in the average. The parameter α was fixed to α = 20, which is identical to the value used in sec. 3.3. |
| Researcher Affiliation | Academia | Tomoyuki Obuchi EMAIL Department of Systems Science Kyoto University Toshiyuki Tanaka EMAIL Department of Systems Science Kyoto University |
| Pseudocode | No | The paper describes mathematical derivations and processes but does not include any explicitly labeled pseudocode or algorithm blocks. The methods are explained in textual and mathematical form. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code for the methodology described, nor does it provide any links to a code repository. |
| Open Datasets | No | Let DM = {(xµ, yµ)}M µ=1 be a dataset of M i.i.d. datapoints following the above data-generation process: P(DM | w0) = µ=1 ryµPΞ|Y. Although the paper mentions general public datasets like "i Naturalist 2018 competition dataset" in the introduction as context, the experiments and analysis in this paper are conducted on a "toy model" where data is generated synthetically according to a defined data-generation process, rather than using an existing public dataset. There is no information provided for accessing the generated data. |
| Dataset Splits | No | The paper uses a toy model with synthetically generated data and performs theoretical analysis verified by numerical simulations. It states: "we conducted simulations with N = 400 and took a sample average over 100 different realizations of the dataset". This describes how multiple datasets are generated and averaged for simulation, but not how a single dataset is split into training, validation, and test sets for model training or evaluation in the traditional sense. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU models, CPU models, or memory specifications used for running the numerical experiments. It only mentions "we conducted simulations with N = 400" which refers to the dimensionality of the model, not hardware. |
| Software Dependencies | No | The paper mentions that "The standard interior-point method was used for the optimization" but does not specify the software or version number of this method or any other software dependencies. |
| Experiment Setup | Yes | For this purpose, we only examined the case with CElo in this section since the CE loss is convex and thus the numerical optimization is relatively easy; such a good property is absent in the zero-one loss. The standard interior-point method was used for the optimization. In the following results, we conducted simulations with N = 400 and took a sample average over 100 different realizations of the dataset; the error bar is the standard error in the average. The parameter α was fixed to α = 20, which is identical to the value used in sec. 3.3. ... Fig. 2 and Fig. 3... (a) Zero-one loss with perceptron ℓ01pe. (b) Cross-entropy loss with logistic function ℓCElo. ... As examples, we compare the balanced case r+ = 0.5 and an imbalanced case r+ = 0.2, with σ = 0.6. ...we examine σ+ = 1, σ = 0.5 with r+ = 0.5 and 0.2 |