Conference on Cognitive Computational Neuroscience

Flexible human behavior requires rapidly updating responses and expectations in the face of changing contingencies or goals. One approach is to identify a latent state, or ‘context’, that governs the relationships between stimuli, responses, and outcomes. While context-dependent decision-making is widely studied, the process by which the human brain learns and represents latent structure remains poorly understood. Here we develop a model of structure learning based on the successor representation (SR), a model of temporal abstraction previously applied to planning and navigation. Unlike in alternative models, ‘context’ is not represented explicitly, but is implied by the temporal relationships between states. We apply the SR to a reversal learning task in which the correct responses and outcomes for a set of stimuli depend on a latent context. Human subjects learn and exploit this structure to infer the properties of all stimuli after a change in just one stimulus. The SR captures the observed learning dynamics at multiple timescales. Moreover, the SR affords a compact, trial-by-trial summary of each subject’s beliefs about task structure that explains individual differences in learning.