Conference on Cognitive Computational Neuroscience

In the hierarchical reinforcement learning framework, complex learning problems are decomposed into sub-components that lead to subgoals, and that can flexibly be combined and used to solve a problem in an absence of immediate rewards. Subgoals are hypothesized to be reinforcing. Little is known about how humans discover useful subgoals, or how they use subgoals to support learning. Here, we show that subgoals reinforce choices even when controlling for other factors often associated with subgoals (such as novelty, bottleneck or external rewards), and that people strategically search in the subgoal space depth-first. Pseudo-reinforcing effect of subgoals also transfers to an independent task.