Conference on Cognitive Computational Neuroscience

Effective decision-making in noisy and changing environments requires balancing exploitation of known rewards with exploration of uncertain options. Yet, the full range of exploratory strategies humans deploy remains unknown, as do the cognitive and neural mechanisms of selection between strategies. Here we use a novel two-armed bandit task, which dissociates exploitation and strategies for directed exploration of different complexity, and fMRI, to study the neural bases of resolving the explore-exploit dilemma. Behaviorally, we find that (i) uncertainty-based exploration has small and idiosyncratic effects on choices, and (ii) exploration is also guided by a simpler reward-based “win-stay-lose-shift” strategy. Neurally, we identify dissociable regions in prefrontal cortex for reward and uncertainty representations implicated in choice, as well as a region in anterior cingulate cortex tracking the use of the heuristic, which may be a candidate neural substrate for strategy selection.