Conference on Cognitive Computational Neuroscience

The exploration-exploitation trade-off, balancing the acquisition of new information with the utilization of known resources, is a fundamental dilemma faced by all adaptive intelligence. Despite our understanding of models based on normative principles, the diverse explore-exploit behaviors of natural intelligence remain elusive. Here, using neural network behavioral modeling and state space analysis, we examined the diverse human exploration behaviors under a novel two-armed bandit task called Changing Bandit, designed to simulate real-world environmental volatility where exploration becomes essential. Examining behavior in the belief state space of this task, we characterized the disparities across artificial agents with decision boundaries. To extend this analysis to human data, a circumstance where choices are too sparse in the belief state space, we trained a recurrent neural network (RNN) model to predict humans’ choices given past observations. This RNN model outperforms all existing cognitive models. Probing the RNN’s decision boundaries, we found substantial individual differences that evade classical cognitive models. Additionally, our RNN revealed a tendency of “high-stay, low-shift” used by humans in response to higher environmental volatilities. Our work offers a promising approach for investigating diverse decision-making strategies in humans and animals.