Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)
Authors: SUBBA REDDY OOTA, Zijiao Chen, Manish Gupta, Bapi Raju Surampudi, Gael Jobard, Frederic Alexandre, Xavier Hinaut
TMLR 2025 | Venue PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our survey dives into this exciting domain, focusing on human brain recording studies and cutting-edge cognitive neuroscience datasets that capture brain activity during natural language processing, visual perception, and auditory experiences. We explore two fundamental approaches: encoding models, which attempt to generate brain activity patterns from sensory inputs; and decoding models, which aim to reconstruct our thoughts and perceptions from neural signals. ... In this survey, we first discuss popular representations of language, vision, and speech stimuli, and present a summary of neuroscience datasets. We then review how the recent advances in deep learning transformed this field, by investigating the popular deep learning based encoding and decoding architectures, noting their benefits and limitations across different sensory modalities. |
| Researcher Affiliation | Collaboration | Subba Reddy Oota EMAIL Inria Bordeaux, France La BRI, Université de Bordeaux, Bordeaux, France Université de Bordeaux, Bordeaux, France TU Berlin, Germany Zijiao Chen EMAIL National University of Singapore, Singapore Manish Gupta EMAIL Microsoft, India International Institute of Information Technology Hyderabad, India Bapi Raju Surampudi EMAIL International Institute of Information Technology Hyderabad, India Gael Jobard EMAIL GIN, IMN-UMR5293, Université de Bordeaux, CEA, CNRS, Bordeaux, France Frederic Alexandre EMAIL Inria Bordeaux, France La BRI, Université de Bordeaux, Bordeaux, France Université de Bordeaux, Bordeaux, France Xavier Hinaut EMAIL Inria Bordeaux, France La BRI, Université de Bordeaux, Bordeaux, France Université de Bordeaux, Bordeaux, France |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. It describes methodologies in text and uses diagrams to illustrate concepts, but no formal pseudocode is presented. |
| Open Source Code | No | Accessing curated ecological stimuli datasets and a Git Hub repository for efficient study initiation 1. 1https://github.com/subbareddy248/Awesome-Brain-Encoding--Decoding. This GitHub repository appears to be a collection of resources for study initiation in the field, not source code for the specific methodologies described or developed within this survey paper. |
| Open Datasets | Yes | In this section, we discuss popular neuroscience datasets that involve audio, visual, and other multimodal stimuli that have been utilized in the literature. Tables 2, 3 and 4 provide detailed overview of types of brain recording, languages used, stimuli presented, number of subjects (|S|), and tasks across datasets of different modalities. ... Publicly available datasets are linked to their sources in the Dataset column. |
| Dataset Splits | No | The paper describes general methods for evaluating models (e.g., using held-out data in a cross-validation setting) and training models (e.g., finding optimal regularization parameters using cross-validation) as commonly done in the field. However, it does not specify concrete dataset split percentages, sample counts, or explicit splitting methodologies for any experiments conducted within this survey paper itself. |
| Hardware Specification | No | The paper is a survey of existing research and does not describe any specific hardware used for running its own experiments. |
| Software Dependencies | No | To visualize the brain maps, popular libraries such as Nilearn 2 or Pycortex 3 are useful for f MRI recordings, while MNE-Python 4 is suitable for both MEG and EEG datasets. ... The minimal preprocessing steps described in f MRIPrep framework (Esteban et al., 2019) ... using MNEPython library (Gramfort et al., 2013)5, the following steps should be executed:. The paper mentions software libraries like Nilearn, Pycortex, MNE-Python, fMRIPrep, but does not provide specific version numbers for these dependencies. |
| Experiment Setup | No | The paper is a survey that discusses experimental setups, hyperparameters, and training settings from other research papers. It does not provide specific experimental setup details for any experiments conducted as part of this paper's own contribution. |