Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1]
Asymptotic Analysis of Objectives Based on Fisher Information in Active Learning
Authors: Jamshid Sourati, Murat Akcakaya, Todd K. Leen, Deniz Erdogmus, Jennifer G. Dy
JMLR 2017 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we attempt to fill this gap and provide a rigorous framework for analyzing existing FIR-based active learning methods. In particular, we show that FIR can be asymptotically viewed as an upper bound of the expected variance of the log-likelihood ratio. Additionally, our analysis suggests a unifying framework that not only enables us to make theoretical comparisons among the existing querying methods based on FIR, but also allows us to give insight into the development of new active learning approaches based on this objective. Keywords: classification active learning, Fisher information ratio, asymptotic log-loss, upper-bound minimization. |
| Researcher Affiliation | Academia | Jamshid Sourati EMAIL Department of Electrical and Computer Engineering Northeastern University Boston, MA 02115 USA Murat Akcakaya EMAIL Department of Electrical and Computer Engineering University of Pittsburgh Pittsburgh, PA 15261 USA Todd K. Leen EMAIL Georgetown University Washington D.C. 20057 USA Deniz Erdogmus EMAIL Jennifer G. Dy EMAIL Department of Electrical and Computer Engineering Northeastern University Boston, MA 02115 USA |
| Pseudocode | Yes | Algorithm 0: Classification with Active Learning Algorithm 1: Fukumizu (2000) Algorithm 2: Zhang and Oles (2000) Algorithm 3: Settles and Craven (2008) Algorithm 4: Hoi et al. (2006, 2009) Algorithm 5: Chaudhuri et al. (2015b) |
| Open Source Code | No | The paper does not provide any statement or link indicating the release of source code for the methodology described. |
| Open Datasets | No | The paper is theoretical and focuses on frameworks and analyses, not on experimental results that would use specific datasets. No dataset is mentioned or made available. |
| Dataset Splits | No | The paper is theoretical and does not conduct experiments with specific datasets, therefore it does not mention dataset splits. |
| Hardware Specification | No | The paper is theoretical and focuses on asymptotic analysis and frameworks. It does not describe any experiments that would require specific hardware, thus no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and describes analytical frameworks and existing algorithms, rather than implementing and running new experiments. Therefore, no specific software dependencies with version numbers are mentioned. |
| Experiment Setup | No | The paper is theoretical, providing an asymptotic analysis of objectives. It does not describe an experimental setup with specific hyperparameters or training configurations. |