P-3A.58

Working Memory Facilitates Reinforcement Learning

Kengo Shibata, Verena Klar, Masud Husain, Sanjay Manohar, University of Oxford, United Kingdom

Session:
Posters 3A Poster

Track:
Cognitive science

Location:
North Schools

Presentation Time:
Sat, 26 Aug, 13:00 - 15:00 United Kingdom Time

Abstract:
Memory and learning have traditionally been studied as separate cognitive processes, but recent research suggests that the neural computations of reinforcement learning (RL) are influenced by working memory (WM). While previous studies have shown a potential trade-off between these two systems, it remains unclear whether WM of the learnt stimuli itself facilitates RL. In two experiments, participants learned action-stimulus-reward mappings while holding the RL stimulus in memory (Exp.1). Although this was not required for learning, encouraging WM activity enhanced RL. Suppressing the memory of the RL stimulus, in contrast (Exp.2), attenuated RL, suggesting that interfering with WM of the RL stimuli disrupts learning. Our results support a model of cognition where RL harnesses representations within WM. This framework produces a number of empirically testable predictions about the interaction of learning and memory.

Manuscript:
License:
Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
DOI:
10.32470/CCN.2023.1447-0
Publication:
2023 Conference on Cognitive Computational Neuroscience
Presentation
Discussion
Resources
No resources available.
Session P-3A
P-3A.1: Development of non-local learning
Alice Zhang, Kate Nussenbaum, Catherine Hartley, New York University, United States
P-3A.2: Cognitive maps at multiple levels of abstraction for flexible inference
Sarah Sweigart, Nam Nguyen, Charan Ranganath, Seongmin Park, Erie Boorman, University of California, Davis, United States
P-3A.3: Internal States and Internal Models Dissociate Components of Motor Beta Oscillations
Tom Marshall, University of Birmingham, United Kingdom; Mengxi Wang, University of Oxford, United Kingdom; Emma Lawrance, Imperial College London, United Kingdom; Nils Kolling, Université Lyon 1, France; Jill O'Reilly, University of Oxford, United Kingdom
P-3A.4: Dosage of transcranial photobiomodulation on working memory in healthy and ADHD adults
Dongwei Li, Jialiang Guo, Beijing Normal University, China; Li Sun, Peking University Sixth Hospital, China; Yan Song, Beijing Normal University, China
P-3A.5: Graded Representations of Economic Value Across Frontal Cortex
Antara Majumdar, Matthias Fritsche, Caitlin Ashcroft, Lauren Strickland, Simon Butt, Armin Lak, University of Oxford, United Kingdom
P-3A.6: Model-Based Assimilation Transmits and Recombines World Models
Ryutaro Uchiyama, Claudio Tennie, Charley Wu, University of Tübingen, Germany
P-3A.7: Spike synchrony as a measure of Gestalt structure
Viktoria Zemliak, Gordon Pipa, Univeristy of Osnabrück, Germany
P-3A.8: Neural feature reconstruction demonstrates cortical re-allocation of working memory representations under unisensory load
Vivien Chopurian, Humboldt Universität zu Berlin, Germany; Simon Weber, Charité Universitätsmedizin Berlin, Germany; Thomas Christophel, Humboldt Universität zu Berlin, Germany
P-3A.9: Dropout as a tool for understanding information distribution in human and machine visual systems
Jacob S. Prince, Harvard University, United States; Gabriel Fajardo, Boston College, United States; George A. Alvarez, Talia Konkle, Harvard University, United States
P-3A.10: Object Real-World Size Representations in Human Brains and Artificial Neural Networks
Zitong Lu, Julie Golomb, The Ohio State University, United States
P-3A.11: Individual Neural Stability on Repeated Episode Viewing Experiences
Menghan Yang, Dartmouth College, United States; Pin-hao (Andy) Chen, National Taiwan University, Taiwan; Amanda Brandt, Eshin Jolly, Luke Chang, Dartmouth College, United States
P-3A.12: Generative model for explaining spatial regularity detection behavior of humans while foraging
Jae Young Jeon, Won Mok Shim, Seng Bum Michael Yoo, 1Ctr. for Neurosci. Imaging Res., Inst. for Basic Sci. (IBS), Suwon, Korea, Republic of 2Dept. of Intelligent Precision Healthcare Convergence, Sungkyunkwan Univ., Suwon, Korea, Republic of 3Dept. of Biomed. Engin., Sungkyunkwan Univ., Suwon, Korea, Republic of, Korea (South)
P-3A.13: Task preparation is reflected in neural state space dynamics
Harrison Ritz, Aditi Jha, Jonathan Pillow, Jonathan Cohen, Princeton University, United States
P-3A.14: A new multi-level modeling framework provides evidence for the simulation of object dynamics in the dorsomedial frontal cortex.
Daniel Calbick, Yale University, United States; Jason Kim, Cornell University, United States; Hansem Sohn, Mehrdad Jazayeri, Massachusetts Institute of Technology, United States; Ilker Yildirim, Yale University, United States
P-3A.15: I See! How Narrative Meaning Influences Gaze Behaviour
Eva Berlot, Lea-Maria Schmitt, Radboud University, Netherlands; Christoph Huber-Huber, University of Trento, Italy; Marius Peelen, Floris de Lange, Radboud University, Netherlands
P-3A.16: Predictive representations explain navigation behavior in multigoal environments
Christoffer Gahnstrom, Russell Epstein, University of Pennsylvania, United States
P-3A.17: A weighted generative model of the human connectome
Danyal Akarca, University of Cambridge, United Kingdom; Simona Schiavi, University of Verona, United Kingdom; Jascha Achterberg, University of Cambridge, United Kingdom; Sila Genc, Derek Jones, Cardiff University, United Kingdom; Duncan Astle, University of Cambridge, United Kingdom
P-3A.18: Response to external feedback is reduced in participants with ASD
Nathaniel Zuk, Yarden Weiss, The Hebrew University, Israel; Athena Akrami, University College London, United Kingdom; Merav Ahissar, The Hebrew University, Israel
P-3A.19: Dynamic Inverse Face Graphics: From 2D Videos to 4D Meshes
Lukas Snoek, Rachael Jack, Philippe Schyns, University of Glasgow, United Kingdom
P-3A.20: Factorization of graphs in the compositional reuse of experience
Lennart Luettgau, University College London, United Kingdom; Rani Moran, University College London; Queen Mary University of London, United Kingdom; Tore Erdmann, Sebastijan Veselic, University College London, United Kingdom; Kimberly L. Stachenfeld, DeepMind, United Kingdom; Zeb Kurth-Nelson, University College London; DeepMind, United Kingdom; Raymond J. Dolan, University College London, United Kingdom
P-3A.21: Next-word prediction is not all you need to align language models and human brains
Gabriele Merlin, Mariya Toneva, Max Planck Institute for Software Systems, Germany
P-3A.22: Deep neural networks optimized for both face detection and face discrimination most accurately predict face-selective neurons in macaque inferior temporal cortex
Kohitij Kar, York University, Canada; Nancy Kanwisher, Massachusetts Institute of Technology, United States; Katharina Dobs, Justus-Liebig University Giessen, Germany
P-3A.23: Minimal condition repetitions required in rapid serial visual presentation decoding paradigms
Tijl Grootswagers, Western Sydney University, Australia
P-3A.24: Zero-Shot Visual Numerical Reasoning in Dual-Stream Neural Networks
Jessica A. F. Thompson, Hannah Sheahan, Christopher Summerfield, University of Oxford, United Kingdom
P-3A.25: Driving and suppressing the human language network using large language models
Greta Tuckute, Aalok Sathe, Shashank Srikant, Maya Taliaferro, Mingye Wang, Martin Schrimpf, Massachusetts Institute of Technology, United States; Kendrick Kay, University of Minnesota, United States; Evelina Fedorenko, Massachusetts Institute of Technology, United States
P-3A.26: Confidence optimally modulates decision policy in reinforcement learning
kobe desender, KU Leuven, Belgium; Tom Verguts, UGent, Belgium
P-3A.27: Predictive coding from compression, control, and recurrence in human brain networks
Dale Zhou, University of Pennsylvania, United States; Ivan Tseytlin, Haverford College, United States; Theodore Satterthwaite, Dani Bassett, University of Pennsylvania, United States
P-3A.28: Auditory Cortex Inhibition Affects Performance in a Sound Lateralization Task
Mafalda Valente, Juan R. Castiñeiras de Saa, Alfonso Renart, Champalimaud Foundation, Portugal
P-3A.29: Features and dynamics of social inference and decision-making in naturalistic human interaction
Nina Rouhani, Ralph Adolphs, John O'Doherty, Tessa Rusch, California Institute of Technology, United States
P-3A.30: Bayesian Causal Inference Underlies Sensory Attenuation in Tactile Perception
Anna-Lena Eckert, Philipps-Universität Marburg, Germany; Elena Fuehrer, Katja Fiehler, Justus-Liebig-Universität Gießen, Germany; Dominik Endres, Philipps-Universität Marburg, Germany
P-3A.31: Bootstrapping compositional generalization with cache-and-reuse
Bonan Zhao, Christopher G. Lucas, Neil R. Bramley, University of Edinburgh, United Kingdom
P-3A.32: DuckSoup: a videoconference experimental platform to transform participants’ voice and face in real-time during social interactions.
Pablo Arias Sarah, University of Glasgow, United Kingdom; Guillaume Denis, Independent researcher, France; Lars Hall, Lund University Cognitive Science, Sweden; Jean-Julien Aucouturier, Femto-ST, CNRS, France; Philippe Schyns, Rachael E. Jack, Petter Johansson, University of Glasgow, United Kingdom
P-3A.33: Comparative Analysis of Visual Motion Perception: Computer Vision Models versus Human Vision
Zitang Sun, Yen-Ju Chen, Yung-Hao Yang, Shin'ya Nishida, Kyoto University, Japan
P-3A.34: Evolving plastic networks that can learn novel cognitive tasks
Thomas Miconi, ML Collective, United States
P-3A.35: Learning and adapting cognitive maps for flexible decision-making
Fabian M. Renz, Shany Grossman, Max Planck Institute for Human Development, Germany; Peter Dayan, MPI for Biological Cybernetics, Germany; Christian F. Doeller, MPI for Human Cognitive and Brain Sciences, Germany; Nicolas W. Schuck, Max Planck Institute for Human Development, Germany
P-3A.36: Predictive Coding Networks for Temporal Prediction
Beren Millidge, Mufeng Tang, University of Oxford, United Kingdom; Mahyar Osanlouy, University of Auckland, New Zealand; Rafal Bogacz, University of Oxford, United Kingdom
P-3A.37: Use of Vector- and Transition-based Strategies is Modulated by Knowledge of the Environment in Human Spatial Planning
Denis Lan, Laurence Hunt, Christopher Summerfield, University of Oxford, United Kingdom
P-3A.38: Learning Cognitive State Representations from Neuronal and Behavioural Data
Akshey Kumar, Research Group Neuroinformatics, Austria; Aditya Gilra, Machine Learning Group, Netherlands; Moritz Grosse-Wentrup, Research Group Neuroinformatics, Austria
P-3A.39: Adaptive History Biases in Perceptual Decisions of Mice
Matthias Fritsche, Antara Majumdar, Lauren Strickland, Samuel Liebana Garcia, Rafal Bogacz, Armin Lak, University of Oxford, United Kingdom
P-3A.40: Combining Different Response Data Modalities for Robust Inference in the Hierarchical Gaussian Filter
Alexander J. Hess, Sandra Iglesias, Stefan Frässle, Laura Köchli, Stephanie Marino, Matthias Müller-Schrader, Jakob Heinzle, Translational Neuromodeling Unit, Institute for Biomedical Engineering, UZH & ETH Zurich, Switzerland; Olivia K. Harrison, University of Otago, New Zealand; Lionel Rigoux, Max Planck Institute for Metabolism Research, Germany; Christoph Mathys, Aarhus University, Denmark; Klaas Enno Stephan, Translational Neuromodeling Unit, Institute for Biomedical Engineering, UZH & ETH Zurich, Switzerland
P-3A.41: A decision-theoretic model of perceptual multistability: perceptual switches as internal actions
Shervin Safavi, Peter Dayan, Max Planck Institute for Biological Cybernetics, Germany
P-3A.42: The Impact of Rarely-firing Nodes in Neural Networks on Representational Geometry and Predictions of Human Similarity Judgments
Nhut Truong, Anna Bavaresco, Uri Hasson, University of Trento, Italy
P-3A.43: Mooney Face Image Processing in a Deep Convolutional Neural Network Compared to Humans
Astrid Zeman, University of Melbourne, Australia; Tim Leers, Hans Op de Beeck, KU Leuven, Belgium
P-3A.44: End-to-end reconstruction of natural images from multi-unit recordings with Brain2Pix
Lynn Le, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Netherlands; Paolo Papale, Antonio Lozano, Netherlands Institute for Neuroscience, Netherlands; Thirza Dado, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Netherlands; Feng Wang, Netherlands Institute for Neuroscience, Netherlands; Marcel van Gerven, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Netherlands; Pieter Roelfsema, Netherlands Institute for Neuroscience, Netherlands; Yağmur Güçlütürk, Umut Güçlü, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Netherlands
P-3A.45: Efficient inverse graphics with a differentiable generative model explains robustness of perception to unusual viewing angles
Hakan Yilmaz, Matthew Muellner, Department of Psychology, Yale University, United States; Joshua B. Tenenbaum, Department of Brain & Cognitive Sciences, MIT, United States; Katharina Dobs, Department of Psychology, Justus-Liebig University Giessen, Germany; Ilker Yildirim, Department of Psychology, Yale University, United States
P-3A.46: Collapsed Inference a Unifying Principle of Attention
Ryan Singh, Christopher L. Buckley, University of Sussex, United Kingdom
P-3A.47: Benefits of synchrony: Improving deep neural networks using complex values and Kuramoto synchronization
Sabine Muzellec, CerCo - CNRS / Brown University, France; Andrea Alamia, CerCo - CNRS, France; Thomas Serre, Brown University, United States; Rufin VanRullen, CerCo - CNRS, France
P-3A.48: Hierarchical Predictive Coding across the Auditory Forebrain
Srihita Rudraraju, Michael Turvey, University of California San Diego, United States; Brad Theilman, Sandia National Laboratories, United States; Timothy Gentner, University of California San Diego, United States
P-3A.49: Human Social Interaction Judgements are Uniquely Explained by both Bottom-up Graph Neural Networks and Generative Inverse Planning Models
Manasi Malik, Leyla Isik, Johns Hopkins University, United States
P-3A.50: Joint Models of Response Times and Raw Visual Stimuli from Speeded Decision-Making Tasks
Paul Jaffe, Stanford University, United States; Robert Schafer, Lumos Labs, United States; Russell Poldrack, Patrick Bissett, Stanford University, United States
P-3A.51: Exploring Cognitive Factors in the Bayesian Pain Model: Precision Modulation and Mean Shifts
Andreas Strube, Christian Büchel, University Medical Center Hamburg-Eppendorf, Germany
P-3A.52: Parvalbumin-Positive Neurons in the Globus Pallidus Externus Modulate Task-Irrelevant Behaviors to Balance Exploration and Exploitation
Minryung Song, Institute of Computational Intelligence Science, Korea (South); Shinwoo Kang, Mayo Clinic College of Medicine and Science, United States; Minsu Yang, Korea Advanced Institute of Science and Technology (KAIST), Korea (South); Robert Bruce, Doo-Sup Choi, Mayo Clinic College of Medicine and Science, United States; Sang Wan Lee, Korea Advanced Institute of Science and Technology (KAIST), Korea (South)
P-3A.53: Habits Through Temporal-Difference Action Learning
Charlotte Collingwood, University of Oxford, United Kingdom; Marcus Stephenson-Jones, University College London, United Kingdom; Rafal Bogacz, University of Oxford, United Kingdom
P-3A.54: The cognitive basis of pain: Arbitration between body-model and world-model pain avoidance learning.
Yijia Yan, Danielle Hewitt, Laurence Hunt, Ben Seymour, University of Oxford, United Kingdom
P-3A.55: The Component Processes of Complex Planning Follow Distinct Developmental Trajectories
Ili Ma, Leiden University, Netherlands; Camille V. Phaneuf, Harvard University, United States; Bas van Opheusden, Princeton University, United States; Wei Ji Ma, Catherine A. Hartley, New York University, United States
P-3A.56: Divergent effects of expectations on behavior and brain
Heejung Jung, Aryan Yazdanpanah, Alireza Soltani, Tor Wager, Dartmouth College, United States
P-3A.57: A biological model of nonlinear dimensionality reduction
Kensuke Yoshida, Taro Toyoizumi, RIKEN Center for Brain Science; The University of Tokyo, Japan
P-3A.58: Working Memory Facilitates Reinforcement Learning
Kengo Shibata, Verena Klar, Masud Husain, Sanjay Manohar, University of Oxford, United Kingdom
P-3A.59: HSSM: Hierarchical Inference for Likelihood Free Cognitive Process Models
Alexander Fengler, Aisulu Omar, Yang Xu, Michael Frank, Brown University, United States
P-3A.60: VIP-Interneurons Control the Effect of Behavioral State on Sensory Responses in Mice Primary Visual Cortex
Ehsan Sabri, Renata Batista-Brito, Albert Einstein College of Medicine, United States