Motivated by the successes of large language models (LLMs), we hypothesized that semantic information in the brain may be expressed in distributed patterns of neural activity. We recorded responses of hippocampal neurons while individuals listened to narrative speech. Here, using an encoding model approach, we find robust encoding of semantics, even after controlling for phonemics and grammar.
Individual words had complex selectivities that included multiple words in multiple semantic categories. Similar to embedding vectors in LLMs, distance between neural population responses correlated with semantic distance. For semantically similar words, even contextual embedders showed an inverse correlation between semantic and neural distances; we attribute this pattern to the noise-mitigating benefits of contrastive coding.
Finally, neural population activity aligned most closely with GPT-2 embeddings, and variation in response patterns correlated with LLM-derived polysemy measures, indicating that semantic encoding is largely contextualized. Ultimately, these results provide a neurocomputational account for understanding how hippocampal neurons encode word meaning during language listening. This is a preview of subscription content, access via your institution Access Nature and 54 other Nature Portfolio journals Get Nature+, our best-value online-access subscription Receive 12 print issues and online access Prices may be subject to local taxes which are calculated during checkout The data that support the findings of this study are available on the Data Archive for the Brain Initiative at https://dabi.loni.usc.edu/projects/W2ZHP46ITR2D.
Source data are provided with this paper. The code underlying this work relies on Python 3.11.7 and Matlab 2023b or higher. The custom code written to analyze the data and generate Figs. 1–4 and Extended Data Figs. 1–10 is available via Github at https://github.com/mcfranch/PopulationCodeForSemanticsHippocampus.
Mitchell, J. & Lapata, M. Vector-based models of semantic composition. ACL-08: HLT (eds Moore, J.
D. et al.) 236–244 (Association for Computational Linguistics, 2008). Decoding semantic representations in mind and brain. Attention is all you need.
Extract — continue reading at the source.
