Test-Time Adaptation for Combating Missing Modalities in Egocentric Videos
Authors: Merey Ramazanova, Alejandro Pardo, Bernard Ghanem, Motasem Alfarra
ICLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through experiments with various pretrained models and datasets, Mi Dl demonstrates substantial performance improvement without the need for retraining. Our code is available in this repo. We thoroughly analyze Mi Dl s performance under different missing modality scenarios. We present our experimental setup in Section 5.1. |
| Researcher Affiliation | Academia | Merey Ramazanova, Alejandro Pardo, Bernard Ghanem, Motasem Alfarra Center of Excellence in Generative AI, KAUST, Saudi Arabia EMAIL |
| Pseudocode | No | The paper describes the method's steps in paragraph text and mathematical equations (Equation 2) but does not include a clearly labeled pseudocode or algorithm block. |
| Open Source Code | Yes | Our code is available in this repo. |
| Open Datasets | Yes | Datasets. Epic-Kitchens (Damen et al., 2018; 2022) contains 100 video hours... Epic-Sounds (Huh et al., 2023) provides sound-based annotations... we leverage the recent Ego4D (Grauman et al., 2022) data. |
| Dataset Splits | Yes | We stick to the official train/val/test splits provided for both datasets. The approximate ratios are 75% for training, 10% for validation, and 15% for testing. The validation set contains 9668 samples. Epic-Sounds ... has 8045 validation samples. |
| Hardware Specification | Yes | Each experiment was run using one V100 GPU. |
| Software Dependencies | No | The paper mentions using SGD as an optimizer but does not specify any software libraries or frameworks with version numbers (e.g., Python, PyTorch, TensorFlow, CUDA versions). |
| Experiment Setup | Yes | Note that we set λ1 = λ2 = 3 for all our experiments. Further, we conduct the adaptation step with an SGD (Robbins & Monro, 1951) step with a learning rate of 25 10 4, and a momentum of 0.9, following (Niu14 et al., 2023; Niu et al., 2022; Wang et al., 2020). We freeze the rest of the model parameters during the update step. |