P-3B.67

Risking your Tail: Curiosity, Danger & Exploration

Tingke Shen, Peter Dayan, Max Planck Institute for Biological Cybernetics, Germany

Session:
Posters 3B Poster

Track:
Cognitive science

Location:
Marquee

Presentation Time:
Sat, 26 Aug, 13:00 - 15:00 United Kingdom Time

Abstract:
Novelty is a double-edged sword for agents and animals alike: they might benefit from untapped resources or face unexpected costs or dangers such as predation. The conventional exploration/exploitation tradeoff is thus coloured by risk-sensitivity. Accordingly, individual differences in exploratory trajectories can reveal the different prior expectations animals have about reward and threat, and different degrees of risk aversion. To capture this setting, we suggest a Bayes adaptive Markov decision process model which has three mechanisms: an adaptive hazard function capturing potential predation, a reward function providing the urge to explore, and a conditional value at risk (CVaR) objective (as a contemporary measure of trait risk-sensitivity). We fit this model to a coarse-grain abstraction of the behaviour of 26 animals recorded as they freely explored a novel object in an open-field arena (Akiti et al. Neuron 110, 2022). We show that it captures both quantitative (frequency, duration of exploratory bouts) and qualitative (stereotyped tail-behind) features of behavior, including the substantial idiosyncrasies that were observed.

Manuscript:
License:
Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
DOI:
10.32470/CCN.2023.1192-0
Publication:
2023 Conference on Cognitive Computational Neuroscience
Presentation
Discussion
Resources
No resources available.
Session P-3B
P-3B.61: Reinforcement learning influences memory specificity across development
Kate Nussenbaum, Catherine Hartley, New York University, United States
P-3B.62: Cross-Task fMRI Decoding: a Window into Mind-Wandering
Ronald Dekker, University of Tokyo, Japan; Amanda Lins, Max Planck Institute, Germany; Aaron Nakamura, Junxia Wang, Morritz Bammel, University of Tokyo, Japan; Quentin Huys, University College London, United Kingdom; Nicolas Schuck, Max Planck Institute, Germany; Mingbo Cai, University of Tokyo, Japan
P-3B.63: A Novel Cognition-guided Neurofeedback Protocol for Methamphetamine Addiction Treatment
Huixing Gou, University of Science and Technology of China, China; Junjie Bu, Anhui Medical University, China; Xiaochu Zhang, University of Science and Technology of China, China
P-3B.64: Age-related differences in latent belief updating and its neural engagement
Yu-Shiang Su, Joshua Oon Soo Goh, National Taiwan University, Taiwan
P-3B.65: Effect of Target-distractor Similarity on Attentional Modulation in the Human Visual Cortex
Narges Doostani, Institute for Research in Fundamental Sciences, Iran; Gholam-Ali Hossein-Zadeh, University of Tehran, Iran; Radoslaw Martin Cichy, Freie Universität Berlin, Germany; Maryam Vaziri-Pashkam, National Institute of Mental Health, United States
P-3B.66: Anticipation of relevant vs. probable content recruits dissociable neural mechanisms
José M. G. Peñalver, Carlos González-García, Ana F. Palenciano, Marta Becerra-Losada, María Ruz, University of Granada, Spain
P-3B.67: Risking your Tail: Curiosity, Danger & Exploration
Tingke Shen, Peter Dayan, Max Planck Institute for Biological Cybernetics, Germany
P-3B.68: Multimodal units extract comodulated information
Marcus Ghosh, Sorbonne Universite, France; Gabriel Bena, Nicolas Perez-Nieves, Imperial College London, United Kingdom; Volker Bormuth, Sorbonne Universite, France; Dan F. M. Goodman, Imperial College London, United Kingdom
P-3B.69: Adapting Learning Rates to Multiple Environments
Jonas Simoens, Senne Braem, Tom Verguts, Ghent University, Belgium
P-3B.70: Humans and Neural Networks Show Similar Patterns of Transfer and Interference in a Continual Learning Task
Eleanor Holton, Lukas Braun, Jessica Thompson, Christopher Summerfield, Oxford University, United States
P-3B.71: Learning Successor Representations in the Hippocampus: Exploring the Role of Temporally Asymmetric and Symmetric Plasticity
Janis Keck, Max Planck Institute for mathematics in the sciences, Max Planck Institute for Human and Cognitive Brain Sciences, Germany; Christian F. Doeller, Max Planck Institute for Human and Cognitive Brain Sciences, Germany; Juergen Jost, Max Planck Institute for Mathematics in the Sciences, Germany; Caswell Barry, University College London, Germany
P-3B.72: When do Measured Representational Distances Reflect the Neural Representational Geometry?
Veronica Bossio Botero, Nikolaus Kriegeskorte, Columbia University, United States
P-3B.73: The Hippocampus sends perceptual predictions to the cortex
Oliver Warrington, Nadine N. Graedel, Martina F. Callaghan, Peter Kok, University College London, United Kingdom
P-3B.74: Privileged representational axes in biological and artificial neural networks
Meenakshi Khosla, Josh McDermott, Nancy Kanwisher, Massachusetts Institute of Technology, United States
P-3B.75: Sensitivity to the instrumental value of agency increases across development
Hanxiao Lu, Perri Katzman, Kate Nussenbaum, Catherine Hartley, New York University, United States
P-3B.76: Certainty-weighted integration of information in individual cortical neurons
Ben von Hünerbein, University of Bern, Switzerland; Matthijs oude Lohuis, Pietro Marchesi, Umberto Olcese, University of Amsterdam, Netherlands; Walter Senn, University of Bern, Switzerland; Cyriel Pennartz, University of Amsterdam, Netherlands; Jakob Jordan, Mihai Petrovici, University of Bern, Switzerland
P-3B.77: Revisiting the Role of Relearning in Semantic Dementia
Devon Jarvis, University of the Witwatersrand, South Africa; Verena Klar, University of Oxford, United Kingdom; Richard Klein, Benjamin Rosman, University of the Witwatersrand, South Africa; Andrew Saxe, University College London, United Kingdom
P-3B.78: CogPonder: Towards a Computational Framework of General Cognitive Control
Morteza Ansarinia, Pedro Cardoso-Leite, University of Luxembourg, Luxembourg
P-3B.79: Evaluating the Role of Edge-Surface Reconstruction in Complex Lightness Illusions
Srijani Saha, George Alvarez, Harvard University, United States
P-3B.80: Simple, Idiosyncratic Decision Heuristics in a Two-Armed Bandit Task
Mirko Thalmann, Eric Schulz, Max Planck Institute for Biological Cybernetics, Germany
P-3B.81: The brain can’t copy-paste: End-to-end topographic neural networks as a way forward for modelling cortical map formation and behaviour
Zejin Lu, Freie Universitaet Berlin, Germany; Adrien Doerig, Victoria Bosch, University of Osnabrueck, Germany; Bas Krahmer, Radboud University, Netherlands; Daniel Kaiser, Justus-Liebig-Universitaet Gießen, Germany; Radoslaw Cichy, Freie Universitaet Berlin, Germany; Tim Kietzmann, University of Osnabrueck, Germany
P-3B.82: Effects of SORL1 rs2070045 polymorphism on cognitive decline and aging-related grey matter pattern in non-demented individuals
Chen Liu, Caishui Yang, Zhanjun Zhang, Beijing Normal University, China
P-3B.83: What did you expect? Prediction error tuning in sensory cortex
David Richter, Donders Institute, Netherlands; Tim Kietzmann, University of Osnabrück, Germany; Floris de Lange, Donders Institute, Netherlands
P-3B.84: Relational Episodic Inference for Fictive Planning
Aleix Alcacer, Marina Martínez Garcia, Universitat Jaume I, Spain; Daniel McNamee, Champalimaud Research, Portugal; Raphael Kaplan, Universitat Jaume I, Spain
P-3B.85: Frequency-Specific Contributions to Perceptual Priors in Audition: Testing the Predictive-Coding Prediction
Aviel Sulem, Itay Lieder, Merav Ahissar, Hebrew University of Jerusalem, Israel
P-3B.86: Functional Double-Dissociation Along the Human Hippocampal Long Axis
Peter A. Angeli, Lauren M. DiNicola, Noam Saadon-Grosman, Harvard University, United States; Randy L. Buckner, Harvard University; Massachusetts General Hospital, United States
P-3B.87: Effect of Environmental Stochasticity on Planning Depth
Jordan Lei, Wei Ji Ma, New York University, United States
P-3B.88: Inverse Reflection Effect During Context-Dependent Learning of Risky Choices
Ali Shiravand, École Normale Supérieure, France; Maëlle Gueguen, Rutgers University-New Brunswick, United States; Sophie Bavard, University of Hamburg, Germany; Julien Bastin, Université Grenoble Alpes, France; Stefano Palminteri, École Normale Supérieure, France
P-3B.89: High-dimensional Sampling in Random Neural Networks Competes With Deep Learning Models of Visual Cortex
Atlas Kazemian, Eric Elmoznino, Michael Bonner, Johns Hopkins University, United States
P-3B.90: First Steps in Using Topographic Deep Artificial Neural Network Models to Generate Hypotheses about Not-yet-detected Functional Neural Clusters in the Ventral Stream
Kamila M. Jozwik, Massachusetts Institute of Technology and University of Cambridge, United Kingdom; Hyodong Lee, Nancy Kanwisher, James J. DiCarlo, Massachusetts Institute of Technology, United States
P-3B.91: Age-Related Changes in Neural Noise in a Decision-Making Task
Fenying Zang, Leiden University, Netherlands; Anup Khanal, University of California Los Angeles, United States; International Brain Laboratory, www.internationalbrainlab.com, United States; Anne K Churchland, University of California Los Angeles, United States; Anne E Urai, Leiden University, Netherlands
P-3B.92: From Primates to Robots: Emerging Oscillatory Latent-Space Dynamics for Sensorimotor Control
Alexander Mitchell, Oiwi Parker Jones, Jun Yamada, Wolfgang Merkt, Ioannis Havoutis, Ingmar Posner, University of Oxford, United Kingdom
P-3B.93: Curriculum Effects on Compositional Generalisation Revisited
Carla Cremer, Jacques Pesnot Lerousseau, Chris Summerfield, Oxford, United Kingdom
P-3B.94: A stimulus-computable model of beta oscillatory responses to speech
Christoph Daube, University of Glasgow, United Kingdom; Joachim Gross, University of Münster, Germany; Robin A. A. Ince, University of Glasgow, United Kingdom
P-3B.95: Explaining the function of the N400 as the update of the internal representations of ANN models of sentence processing
Alessandro Lopopolo, Milena Rabovsky, University of Potsdam, Germany
P-3B.96: Mechanisms of Input-Frequency Dependent Pattern Separation in the Dentate Gyrus
Selena Singh, Suzanna Becker, McMaster University, Canada; Thomas Trappenberg, Abraham Nunes, Dalhousie University, Canada
P-3B.97: Reward morphs non-spatial cognitive maps in humans
Nir Moneta, Max Planck Institute for Human Development Berlin, Germany; Charley M. Wu, University of Tübingen, Germany; Christian F. Doeller, Max Planck Institute for Human Cognitive and Brain Sciences, Germany; Nicolas W. Schuck, Universität Hamburg, Germany
P-3B.98: Does Posner cueing engage attention or expectation? Answers from an embedding-filtered deep convolutional network
Mainak Biswas, Barath Mohan Umapathi, Sricharan Sunder, Devarajan Sridharan, Indian Institute of Science, India
P-3B.99: Uncertainty-driven exploration in the basal ganglia
Yuhao Wang, Armin Lak, Sanjay Manohar, Rafal Bogacz, University of Oxford, United Kingdom
P-3B.100: Modelling Novelty Detection in the Cortex with Predictive Coding
Tianjin Li, Mufeng Tang, Rafal Bogacz, University of Oxford, United Kingdom
P-3B.101: Teasing apart the representational spaces of ANN language models to discover key axes of model-to-brain alignment
Eghbal Hosseini, Noga Zaslavsky, Colton Casto, Evelina Fedorenko, Massachusetts Institute of Technology, United States
P-3B.102: Two-dimensional Reward Evaluation and Its Relevance to Anhedonia
Yan Yan, Stanford University, United States; Margaret Westwater, Laurence Hunt, Michael Browning, University of Oxford, United Kingdom
P-3B.103: Higher-level spatial prediction during natural scene perception in mouse primary visual cortex
Micha Heilbron, Floris de Lange, Donders Institute for Brain, Cognition, and Behaviour, Radboud University, Netherlands
P-3B.104: Generalization and Discrimination in Reinforcement Learning: Developmental Trajectories and the Potential Link with Psychotic Symptoms
Wei Chen, Aaron Nakamura, The University of Tokyo, Japan; Jialing Ding, KU Leuven, Belgium; Naohiro Okada, Shinsuke Koike, Sho Yagishita, Shin Ishii, Haruo Kasai, Yuko Yotsumoto, Ming Bo Cai, The University of Tokyo, Japan
P-3B.105: The Role of Habituation in Risk-taking Escalation
Hadil Haj Ali, Moshe Glickman, University College London, The Max Planck UCL Centre for Computational Psychiatry and Ageing Research, United Kingdom; Tali Sharot, University College London, The Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Massachusetts Institute of Technology, United Kingdom
P-3B.106: The size-weight illusion is explained by efficient coding based on correlated natural statistics
Paul Bays, University of Cambridge, United Kingdom
P-3B.107: Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamic Mode Representational Similarity Analysis
Mitchell Ostrow, Adam Eisen, Leo Kozachkov, Ila Fiete, Massachusetts Institute of Technology, United States
P-3B.108: A shared neural circuit for maintenance and integration of information over time
Peter Murphy, Maynooth University, Ireland; Hannah McDermott, Freie University, Germany; Klaus Wimmer, Centre de Recerca Matemàtica, Spain; Jade Duffy, Trinity College Dublin, Ireland; Jose Esnaola-Acebes, Centre de Recerca Matemàtica, Spain; Albert Compte, Institut D’Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Spain; Robert Whelan, Trinity College Dublin, Ireland
P-3B.109: A Neuro-symbolic Model of Event Comprehension
Tan Nguyen, Matthew Bezdek, Washington University in St. Louis, United States; Samuel Gershman, Harvard University, United States; Todd Braver, Aaron Bobick, Jeffrey Zacks, Washington University in St. Louis, United States
P-3B.110: Spatially-embedded recurrent spiking neural networks reveal patterns of topologically structured computations
Andrew Siyoon Ham, Duncan E. Astle, University of Cambridge, United Kingdom; Jascha Achterberg, University of Cambridge / Intel Labs, United Kingdom; Danyal Akarca, University of Cambridge, United Kingdom
P-3B.111: AI-driven cholinergic theory enables rapid and robust cortex-wide learning
Maija Filipovica, Kevin Kermani Nejad, Will Greedy, Heng Wei Zhu, Jack Mellor, Rui Ponte Costa, University of Bristol, United Kingdom
P-3B.112: Characterising the spatiotemporal profiles of neural object representations using implicit and explicit similarity judgement tasks
Peter Brotherwood, Université de Montréal, Canada; Simon Faghel-Soubeyrand, University of Oxford, United Kingdom; Jasper van den Bosch, Ian Charest, Université de Montréal, Canada
P-3B.113: Illusions of Confidence in Artificial Systems
Clara Colombatto, Steve Fleming, University College London, United Kingdom
P-3B.114: Decoding accuracies as well as ERP amplitudes do not show between-task correlations
Benedikt V. Ehinger, Hannes Bonasch, University of Stuttgart, Germany
P-3B.115: Planning with Others in Mind
Nastaran Arfaei, WeiJi Ma, NYU, United States
P-3B.116: A Representation of Event Probability Density in Temporo-Parietal Neural Dynamics
Matthias Grabenhorst, David Poeppel, Ernst Strüngmann Institute for Neuroscience in Cooperation with Max Planck Society, Germany; Georgios Michalareas, Max Planck Institute for Empirical Aesthetics, Germany
P-3B.117: Neural Representations of Algorithms in the Logical Reasoning Network are Recycled for Programming Code Comprehension
Yun-Fei Liu, Janice Chen, Colin Wilson, Marina Bedny, Johns Hopkins University, United States
P-3B.118: Projectional motifs facilitate sequence memorization and transfer
Shuchen Wu, Mirko Thalmann, Eric Schulz, Max Planck Institute for Biological Cybernetics, Germany
P-3B.119: Hyper-HMM: simultaneous temporal and spatial pattern alignment for brains and stimuli
Caroline Lee, Columbia University, Dartmouth College, United States; Jane Han, Ma Feilong, Guo Jiahui, James Haxby, Dartmouth College, United States; Christopher Baldassano, Columbia University, United States
P-3B.120: Skip Connections Increase the Capacity of Variable Binding Mechanisms
Yi Xie, Massachusetts Institute of Technology, United States; Yichen Li, Harvard University, United States; Tomaso Poggio, Akshay Rangamani, Massachusetts Institute of Technology, United States