Rethinking High-speed Image Reconstruction Framework with Spike Camera
Authors: Kang Chen, Yajing Zheng, Tiejun Huang, Zhaofei Yu
AAAI 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct quantitative and qualitative analyses on the U-CALTECH and U-CIFAR datasets, with experiments showing that our proposed Spike CLIP achieves remarkable restoration performance under low-light conditions. Furthermore, the reconstructed images are well-aligned with the broader visual features needed for downstream tasks, ensuring more robust and versatile performance in challenging environments. |
| Researcher Affiliation | Academia | 1School of Computer Science, Peking University 2State Key Laboratory for Multimedia Information Processing, Peking University 3Institute for Artificial Intelligence, Peking University EMAIL, EMAIL |
| Pseudocode | No | The paper describes the framework and methods in detail using natural language and figures (e.g., Figure 2 for the overall framework, Figure 3 for HQ images generation pipeline), but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code https://github.com/chenkang455/Spike CLIP |
| Open Datasets | Yes | We conduct both quantitative and qualitative experiments on the UHSR dataset (Zhao et al. 2024a), which includes realworld spikes captured by the spike camera under low light from ultra-high-speed moving objects of the CALTECH (Fei-Fei, Fergus, and Perona 2004) and CIFAR (Krizhevsky, Hinton et al. 2009) datasets, referred to as U-CALTECH and U-CIFAR respectively. |
| Dataset Splits | Yes | Each dataset contains 5,000 spike-label training pairs and 1,000 test pairs, with 250 250 spatial resolution. |
| Hardware Specification | Yes | the overall experimental framework is built on the Py Torch platform and trained based on an NVIDIA 4090 GPU. |
| Software Dependencies | No | The paper states that "the overall experimental framework is built on the Py Torch platform" but does not specify the version number for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | We employ the Adam optimizer with a learning rate of 4e-5 on the UCALTECH dataset and 1e-4 on the U-CIFAR dataset. The training process consists of 25 total epochs: 5 epochs for coarse reconstruction, 1 epoch for prompt optimization, and 19 epochs for fine reconstruction. We employ the voxelization technique to convert the spike stream from an initial length of 200 to 50. |