Conference on Cognitive Computational Neuroscience

The timing of semantic processing during object recognition in the brain remains a topic of ongoing discussion. One way of addressing this question is by applying multivariate pattern analysis (MVPA) to human electrophysiological responses to object images of different semantic categories. However, while MVPA can reveal whether neuronal activity patterns are distinct for different stimulus categories, concerns remain on whether low-level visual features also contribute to the classification results. To address this issue, we applied a cross-decoding approach to MEG data from stimuli from two different modalities: images and written words. Furthermore, we combined MVPA with a time-delay embedding technique used to characterize dynamical systems from measurements with limited resolution. Our results provide evidence for semantic co-activation between textual and pictorial occurring as early as 150-200 ms. This early onset of semantic processing is surprising in relation to previous work employing e.g. the N400 paradigm estimating the onset of semantic processing to ≥250ms.