Conference on Cognitive Computational Neuroscience

Although much progress in understanding the human brain has been driven by task-based neuroimaging studies, a substantial proportion of mental processes occurs off-task in the form of spontaneous thought. The covert nature of spontaneous thought presents a challenge to studying its underlying processes and dynamics. To overcome this challenge, we developed a computational pipeline for cross-task semantic decoding, in particular from movie-watching to spontaneous thought. We recorded fMRI from 17 subjects who watched a movie, recalled scenes and were scanned during rest. We use the representational space of a large language model as our decoding target, and introduce a method to ameliorate loss induced by cross-task decoding. Our results show that spontaneous thought can be characterized as a series of transitions between latent states in a hidden Markov model with interpretable content and transition parameters, which may have practical applications in the diagnosis and treatment of mental disorders.