Conference on Cognitive Computational Neuroscience

Expectations, derived from prior knowledge, shape our perception of the world and facilitate behavior. A central framework for thinking about the underlying neural mechanisms is predictive coding, i.e. the brain inhibits expected sensory input while facilitating surprise. Indeed, neural activity in response to surprising stimuli is often shown to be enhanced. What remains unknown, however, is the nature of the code representing surprise. Do the elevated responses represent a local code, such as oriented edges in early visual areas, or is surprise instead represented in terms of higher-level features relayed top-down? Here we arbitrated between these hypotheses using fMRI and visual feature dissimilarity metrics derived from a deep neural network model. We show that prediction errors in the visual system, including early visual cortex, scale with high-level visual feature surprise. This suggests that visual areas inherit predictions top-down and hence prediction errors in early areas reflect higher visual feature tuning.