ORCA: A Matlab/Octave Toolbox for Ordinal Regression
Authors: Javier Sánchez-Monedero, Pedro A. Gutiérrez, María Pérez-Ortiz
JMLR 2019 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | ORCA (Ordinal Regression and Classification Algorithms) is a Matlab/Octave framework that implements and integrates different ordinal classification algorithms and specifically designed performance metrics. The framework simplifies the task of experimental comparison to a great extent, allowing the user to: (i) describe experiments by simple configuration files; (ii) automatically run different data partitions; (iii) parallelize the executions; (iv) generate a variety of performance reports and (v) include new algorithms by using its intuitive interface. |
| Researcher Affiliation | Academia | Javier S anchez-Monedero EMAIL School of Journalism, Media and Culture CardiffUniversity CardiffCF10 1FS, Wales, United Kingdom Pedro A. Guti errez EMAIL Department of Computer Science and Numerical Analysis University of C ordoba C ordoba, 14071, Spain Mar ıa P erez-Ortiz EMAIL Department of Computer Science University College London London WC1E 6EA, United Kingdom |
| Pseudocode | No | The paper includes 'Sample Code' in Section 4, which provides Matlab/Octave code snippets demonstrating API usage. While formatted like code, it is presented as executable example code rather than a generalized algorithm block or pseudocode explicitly detailing a method's steps. |
| Open Source Code | Yes | Source code, binaries, documentation, descriptions and links to data sets and tutorials (including examples of educational purpose) are available at https://github.com/ayrna/orca. |
| Open Datasets | Yes | Source code, binaries, documentation, descriptions and links to data sets and tutorials (including examples of educational purpose) are available at https://github.com/ayrna/orca. We also provide several small data sets for code testing purposes and a list of 44 ordinal data sets of different characteristics that can be used for algorithm comparison. |
| Dataset Splits | Yes | It processes every experiment by fitting a model with the training data (including model selection through cross-validation), evaluating the test error and creating performance reports of all the performance metrics... For example, if we perform 30 repetitions of a hold-out design for 10 data sets, we need at least 300 calls to the fit and predict functions, which can be run in parallel. |
| Hardware Specification | No | The paper mentions 'Matlab and Octave parallelisation toolboxes' and 'HTCondor distributed computing environment' for running experiments, but does not provide specific hardware details such as CPU/GPU models, memory, or specific cloud instance types. |
| Software Dependencies | No | The paper states 'ORCA (Ordinal Regression and Classification Algorithms) is a Matlab/Octave framework' and mentions using 'Matlab and Octave parallelisation toolboxes' and a 'Matlab wrapper for LIBLINEAR'. However, no specific version numbers are provided for Matlab, Octave, their toolboxes, or LIBLINEAR. |
| Experiment Setup | No | The paper mentions 'a specific hyper-parameter configuration of the selected classifier' and that the framework records 'hyper-parameter values'. It also refers to 'model selection through cross-validation'. However, no concrete hyperparameter values (e.g., learning rate, batch size, specific optimizer settings, number of epochs) are explicitly provided in the main text. |