A Neuro-symbolic Model of Event Comprehension
Tan Nguyen, Matthew Bezdek, Washington University in St. Louis, United States; Samuel Gershman, Harvard University, United States; Todd Braver, Aaron Bobick, Jeffrey Zacks, Washington University in St. Louis, United States
Session:
Posters 3B Poster
Presentation Time:
Sat, 26 Aug, 13:00 - 15:00 United Kingdom Time
Abstract:
Humans segment continuous experience into discrete events and learn the shared structure of events to predict how activity will unfold. While theoretical accounts provide valuable insights, a computational model offers detailed predictions and a framework to test mechanistic hypotheses. A neuro-symbolic model combining probabilistic reasoning to infer latent classes of events with gated neural networks to learn short-term event dynamics was trained on a large-scale corpus of human activity sequences. Two mechanisms at the symbolic level were evaluated to model transitioning between event classes: prediction error and prediction uncertainty. Both variants successfully learned to predict the unfolding of the activities, and segmented the behavior stream into units and categories similarly to human observers. Model comparison suggested that prediction uncertainty captured human segmentation better than prediction error.