Title: Shared computational principles for language processing in humans and deep language models

Abstract: Naturalistic experimental paradigms in neuroimaging arose from a pressure to test the validity of models we derive from highly controlled experiments in real-world contexts. In many cases, however, such efforts led to the realization that models developed under particular experimental manipulations failed to capture much variance outside the context of that manipulation. The critique of non-naturalistic experiments is not a recent development; it echoes a persistent and subversive thread in the history of modern psychology. The brain has evolved to guide behavior in a multidimensional world with many interacting variables. The assumption that artificially decoupling and manipulating these variables will lead to a good understanding of the brain may be untenable.

Recent advances in artificial neural networks provide an alternative computational framework to model cognition in natural contexts. In contrast to the simplified and interpretable hypotheses we test in the lab, these models do not learn simple, human-interpretable rules or representations of the world. Instead, they use local computations to interpolate over task-relevant manifolds in high-dimensional parameter space. Counterintuitively, over-parameterized deep neural models are parsimonious and straightforward, as they provide a versatile, robust solution for learning diverse functions in natural contexts. Naturalistic paradigms should not be deployed as an afterthought if we hope to build models of brain and behavior that extend beyond the laboratory into the real world.