Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint
Authors: Guangkun Nie, Gongzheng Tang, Shenda Hong
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments across three datasets spanning computer vision and healthcare: IMDB-WIKI-DIR, Age DB-DIR, and ECG-KDIR. The results demonstrate that Dist Loss effectively mitigates the impact of imbalanced data distributions, achieving state-of-the-art performance in few-shot regions. |
| Researcher Affiliation | Academia | Guangkun Nie1,2,*, Gongzheng Tang1,2,*, Shenda Hong1,2,3, 1 Institute of Medical Technology, Health Science Center of Peking University, Beijing, China 2 National Institute of Health Data Science, Peking University, Beijing, China 3 Institute for Artificial Intelligence, Peking University, Beijing, China EMAIL, EMAIL,EMAIL |
| Pseudocode | No | The paper describes the proposed method in section 3.2 'Dist Loss' and illustrates it with Figure 2, but it does not contain a clearly labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | To facilitate further research, we provide our implementation at https://github.com/Ngk03/DIR-Dist-Loss. |
| Open Datasets | Yes | We evaluated our method on three datasets, focusing on tasks of age estimation and potassium concentration prediction. The IMDI-WIKI-DIR dataset Yang et al. (2021), derived from the IMDB-WIKI dataset Rothe et al. (2018), consists of 213,553 facial image pairs annotated with age information... The Age DB-DIR dataset Yang et al. (2021), derived from the Age DB dataset Moschoglou et al. (2017), comprises 16,488 facial image pairs with age annotations... The ECG-K-DIR dataset, sourced from the MIMIC-IV dataset Johnson et al. (2020), includes 375,745 pairs of single-lead ECG signals paired with potassium concentration values. |
| Dataset Splits | Yes | The IMDI-WIKI-DIR dataset... This dataset is partitioned into 191,509 samples for training, 11,022 for validation, and 11,022 for testing. The Age DB-DIR dataset... It is divided into 12,208 samples for training, 2,140 for validation, and 2,140 for testing. The ECG-K-DIR dataset... This dataset is divided into 365,549 samples for training, 5,098 for validation, and 5,098 for testing. For the IMDB-WIKI-IR and Age DB-DIR datasets, we maintain consistency with previous studies, where few/median/many correspond to areas with fewer than 20, between 20-100, and more than 100 samples, respectively. For the ECG-K-DIR dataset, assuming that the maximum number of samples for a single label is nmax, we define areas with more than 0.5 nmax, between 0.15-0.5 nmax, and fewer than 0.15 nmax samples as many/median/few shots areas, respectively. |
| Hardware Specification | Yes | We trained all models on the IMDB-WIKI-DIR and Age DB-DIR datasets using a single NVIDIA Ge Force RTX 3090 GPU and on the ECG-K-DIR dataset using a single NVIDIA Ge Force RTX 4090 GPU. |
| Software Dependencies | No | The paper mentions using the Adam optimizer and network architectures like ResNet-50 and Net1D, but does not specify version numbers for any software libraries, programming languages, or development environments. |
| Experiment Setup | Yes | On the IMDB-WIKI-DIR dataset, we selected Res Net-50 as the network architecture. During training, the training epochs were set to 90, with an initial learning rate of 0.001, which was reduced to 1/10 of its value at the 60th and 80th epochs. We employed the Adam optimizer with a momentum of 0.9 and a weight decay of 0.0001. For our method and Balanced MSE, we used a batch size of 512. ... On the ECG-K-DIR dataset, we utilized the Res Net variant, Net1D Hong et al. (2020), as our network architecture. The training was set for 10 epochs with an initial learning rate of 0.001, which was reduced to 1/10 of its initial value at the 5th and 8th epochs. We employed the Adam optimizer with a momentum of 0.9 and a weight decay of 0.00001. A batch size of 512 was used for all methods. |