On Tail Decay Rate Estimation of Loss Function Distributions
Authors: Etrit Haxholli, Marco Lorenzi
JMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We test CTE in a series of experiments on simulated and real data1, showing the improved robustness and quality of tail estimation as compared to classical approaches. Keywords: Extreme Value Theory, Tail Modelling, Peaks-Over-Threshold, Cross-Tail Estimation, Model Ranking. 5. Experiments In this section, we demonstrate the significance of Theorem 21. In the first subsection, we show experimental evidence that the estimated shape parameter of the marginal distribution, coincides with the maximal shape parameter of individual conditional distributions. In the second subsection, we show that when the sample size is finite, as it is the case in practice, the method proposed by Theorem 21 (cross tail estimation) can be necessary to reduce the required sample size for proper tail shape parameter estimation of marginal distributions. Furthermore, in the third subsection, we compare the standard POT and cross tail estimation on real data. |
| Researcher Affiliation | Academia | Etrit Haxholli EMAIL Epione Research Group Inria, Univesity Cte d Azur 2004 Rte des Lucioles, 06902 Valbonne, France. Marco Lorenzi EMAIL Epione Research Group Inria, Univesity Cte d Azur 2004 Rte des Lucioles, 06902 Valbonne, France |
| Pseudocode | Yes | Algorithm 1 Naive Cross Tail Estimation. Algorithm 2 Cross Tail Estimation. Algorithm 3 Construction of a Continuous Mixture Distribution and Direct POT Usage. Algorithm 4 Modification of Algorithm 3 to Ensure High Location Variability. Algorithm 5 Application of CTE on the Mixture Distribution Defined in Algorithm 4. |
| Open Source Code | Yes | 1. The code is available at https://github.com/ehaxholli/CTE |
| Open Datasets | Yes | In this experiment, our data is composed of a one-dimensional time series taken from the UCR Time Series Anomaly Archive 2 (Wu and Keogh, 2020)... 2. https://www.cs.ucr.edu/~eamonn/time_series_data_2018/UCR_Time Series Anomaly Datasets2021. |
| Dataset Splits | Yes | On each run we randomly select 340 points of D for training (denote Di), and then group the predictions of the model on the 1e4 points of D into an array which we denote by ˆYi. Then we split ˆYi into five equally sized subsets ˆYi,j. |
| Hardware Specification | No | No specific hardware details (like GPU/CPU models, memory, or cloud instance types) are provided in the paper. The acknowledgments mention "the OPAL infrastructure from Universit Cte d Azur for providing resources and support," but this is not specific enough to meet the criteria. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., library names with versions, programming language versions) are mentioned in the paper. |
| Experiment Setup | Yes | These experiments are repeated for length scale parameters given in the x axis of Figure 4 as well as in Appendix H. We repeat every experiment 200 times to account for variability across different runs, we compute the mean and standard deviation of the results. In Section 5.3.2 'Polynomial Kernels', it is mentioned: 'We test polynomial kernels of degree from 1 to 9'. |