Conference on Cognitive Computational Neuroscience

Novelty is a double-edged sword for agents and animals alike: they might benefit from untapped resources or face unexpected costs or dangers such as predation. The conventional exploration/exploitation tradeoff is thus coloured by risk-sensitivity. Accordingly, individual differences in exploratory trajectories can reveal the different prior expectations animals have about reward and threat, and different degrees of risk aversion. To capture this setting, we suggest a Bayes adaptive Markov decision process model which has three mechanisms: an adaptive hazard function capturing potential predation, a reward function providing the urge to explore, and a conditional value at risk (CVaR) objective (as a contemporary measure of trait risk-sensitivity). We fit this model to a coarse-grain abstraction of the behaviour of 26 animals recorded as they freely explored a novel object in an open-field arena (Akiti et al. Neuron 110, 2022). We show that it captures both quantitative (frequency, duration of exploratory bouts) and qualitative (stereotyped tail-behind) features of behavior, including the substantial idiosyncrasies that were observed.