Conference on Cognitive Computational Neuroscience

Models of brain organization assume that retinotopic coding is replaced by non-retinotopic, abstract coding as information propagates through the visual hierarchy towards memory structures. This view raises a puzzle for constructive accounts of visual memory: how can mnemonic and visual information interact in the brain if they are represented in fundamentally different reference frames? Here, we used an encoding model of population receptive fields with fMRI to test the hypothesis that visuospatial coding persists in mnemonic cortex and plays a role in structuring mnemonic-perceptual interactions between brain areas. We found that category-selective mnemonic areas anterior to classically defined visual areas contain visually evoked population receptive fields, pRFs. Strikingly, the transition from perceptual to mnemonic cortex is marked by the emergence of inverted visually evoked responses (i.e., positive to negative pRFs). These +/- pRF populations in perceptual and mnemonic cortex, respectively, are closely matched in their visual field representations and exhibit spatially-specific opponent responses during both top-down memory recall and bottom-up visual processing. Together, these results show that retinotopy provides a common reference frame aligning perceptual and mnemonic processes, thereby scaffolding their dynamic interplay.