On the Learnability of Out-of-distribution Detection
Authors: Zhen Fang, Yixuan Li, Feng Liu, Bo Han, Jie Lu
JMLR 2024 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | To study the generalization of OOD detection, this paper investigates the probably approximately correct (PAC) learning theory of OOD detection that fits the commonly used evaluation metrics in the literature. First, we find a necessary condition for the learnability of OOD detection. Then, using this condition, we prove several impossibility theorems for the learnability of OOD detection under some scenarios. Although the impossibility theorems are frustrating, we find that some conditions of these impossibility theorems may not hold in some practical scenarios. Based on this observation, we next give several necessary and sufficient conditions to characterize the learnability of OOD detection in some practical scenarios. Lastly, we offer theoretical support for representative OOD detection works based on our OOD theory. |
| Researcher Affiliation | Academia | Zhen Fang EMAIL Australian Artificial Intelligence Institute University of Technology Sydney 61 Broadway, Ultimo NSW 2007, Australia; Yixuan Li EMAIL Department of Computer Sciences The University of Wisconsin Madison 1210 W Dayton St, Madison, WI 53706, USA; Feng Liu EMAIL School of Computing and Information Systems The University of Melbourne 700 Swanston Street, Carlton VIC 3053, Australia; Bo Han EMAIL Department of Computer Science Hong Kong Baptist University Kowloon Tong, Hong Kong SAR; Jie Lu EMAIL Australian Artificial Intelligence Institute University of Technology Sydney 61 Broadway, Ultimo NSW 2007, Australia |
| Pseudocode | No | The paper focuses on theoretical proofs, lemmas, and theorems related to the learnability of Out-of-Distribution (OOD) detection. It does not include any structured pseudocode or algorithm blocks describing a specific implementation. |
| Open Source Code | No | The paper does not contain any statements about releasing source code, nor does it provide any links to code repositories for the methodology described. |
| Open Datasets | No | The paper is theoretical and does not conduct experiments that would require providing concrete access information to datasets. While it mentions benchmark datasets like 'MNIST', 'CIFAR-10', and 'ImageNet' in the 'Discussion' section to provide context for 'far-OOD' and 'near-OOD' detection, it does not use these datasets for its own empirical validation or provide access details in the context of its own methodology. |
| Dataset Splits | No | The paper is theoretical and does not conduct experiments that would involve dataset splits. Therefore, it does not provide specific information about training, test, or validation splits. |
| Hardware Specification | No | The paper is purely theoretical and does not describe any experiments that would require specific hardware. Therefore, no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and does not describe any experiments or implementations that would require specific software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and focuses on the learnability of OOD detection. It does not describe any experimental setups, hyperparameters, or training configurations for an implementation. |