Function Encoders: A Principled Approach to Transfer Learning in Hilbert Spaces
Authors: Tyler Ingebrand, Adam Thorpe, Ufuk Topcu
ICML 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We propose a method grounded in the theory of function encoders to achieve all three types of transfer. Specifically, we introduce a novel training scheme for function encoders using leastsquares optimization, prove a universal approximation theorem for function encoders, and provide a comprehensive comparison with existing approaches such as transformers and metalearning on four diverse benchmarks. Our experiments demonstrate that the function encoder outperforms state-of-the-art methods on four benchmark tasks and on all three types of transfer. |
| Researcher Affiliation | Academia | 1Oden Institute for Computational Engineering and Sciences, The University of Texas at Austin, Austin, TX, USA 2Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA 3Department of Aerospace Engineering and Engineering Mechanics, The University of Texas at Austin, Austin, TX, USA. Correspondence to: Tyler Ingebrand <EMAIL>. |
| Pseudocode | Yes | Algorithm 1 Function Encoder Training (LS) given source datasets {DS1, . . . , DSn}, learning rate α Initialize basis {g1, . . . , gk} with parameters θ while not converged do for all DSℓ {DS1, . . . , DSn} do ... |
| Open Source Code | No | Project page: tyler-ingebrand.github.io/FEtransfer. The provided URL is a project page, which does not constitute a direct link to a source-code repository as per the instructions. |
| Open Datasets | Yes | We consider four benchmark transfer tasks: 1) a polynomial regression task to illustrate the proposed categories, 2) a CIFAR image classification task, 3) an inference task on the 7-Scenes dataset, and 4) a dynamics estimation task on Mu Jo Co data. We evaluate the function encoder and relevant baselines on the CIFAR 100 dataset (Krizhevsky, 2009). We use the 7-Scenes dataset (Shotton et al., 2013). We adapt the hidden-parameter Mu Jo Co Ant dataset from Ingebrand et al. (2024a). |
| Dataset Splits | Yes | 90 classes are used during training, and 10 are held out for testing. Six scenes are used for training, and one is heldout. We use 10,000 episodes of length 1,000 for training. The first 200 transitions are used as example data, and the next 800 are query points. |
| Hardware Specification | No | No specific hardware details (like GPU/CPU models or memory) are mentioned in the paper. |
| Software Dependencies | No | No specific software dependencies with version numbers are mentioned in the paper. |
| Experiment Setup | Yes | Algorithm 1 Function Encoder Training (LS) given source datasets {DS1, . . . , DSn}, learning rate α Initialize basis {g1, . . . , gk} with parameters θ while not converged do... Additionally, as is common for image based classification problems, weight-regularization is necessary to prevent overfitting for some algorithms, including the function encoder. In addition to the standard baselines, we also use two ad-hoc baselines: Siamese networks (Bromley et al., 1993) and prototypical networks (Snell et al., 2017). MAML additionally required hyper-parameter tuning, especially the internal learning rate. |