A prevailing view holds that although neuroscience historically inspired AI, its role in the recent evolution of AI, driven by the scaling of foundation models, is minor. We challenge this perspective by establishing a biological analogy between simplified dendritic neuron models and the attention mechanisms central to AI foundation models. We show that AI attention mechanisms bear direct mappings to internal neuronal complexities arising from dendritic organization, rather than to the mere inter-neuronal connections of simple point neurons.
This re-conceptualization not only offers a neuroscience-informed perspective for understanding core AI methodologies beyond their formulation as abstract mathematical operations, but also paves the way for neuro-inspired foundation models that leverage neuroscientific insights to sustainably advance modern AI beyond scaling laws. As an example of this paradigm, we introduce a dendritic-inspired complexity framework, and design a hybrid-attention foundation model with dendritic-inspired sparse computation, which enhances memory capacity while reducing computational demands. Our model supports a 50M context window-equivalent to roughly 40 volumes of Shakespeare-demonstrating superior long-term memory capabilities.
This work pioneers a paradigm that harnesses intra-neuronal dynamics to drive sustained advances in modern AI, while also offering a potential route to understanding dendritic computational mechanisms. We would like to thank Prof. Giacomo Indiveri for insightful discussions.
This work was partially supported by the National Natural Science Foundation of China (62236009, U22A20103, U2541222, 62406322), the National Distinguished Young Scholars (62325603), the CAS Project for Young Scientists in Basic Research (YSBR-116), the Brain Science and Brain-like Intelligence Technology - National Science and Technology Major Project (2025ZD0215500), the Beijing Science and Technology Plan (Z241100004224011), the Brain-Intelligence Integration Young Scientist Project (BI202507), and the CAAI-Tencent Rhino-Bird Open Research Fund. These authors contributed equally: Man Yao, Yuqi Pan, Yiran Zhong, Shijie Mei, Zhen Qin, Kexin Wang. Institute of Automation, Chinese Academy of Sciences, Beijing, China Man Yao, Yuqi Pan, Shijie Mei, Kexin Wang, Jiakui Hu, Siyu Ding, Tianxiang Hu, Yanbin Huang, Siyuan Xu, Yuhong Chou, Ning Qiao, Bo Xu & Guoqi Li State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Beijing, China Man Yao, Ning-long Xu, Yachuang Hu & Guoqi Li Token Foundry, Alibaba Group, Shanghai, China Institute of Medical Technology, Peking University Health Science Center, Peking University, Beijing, China Center for Brain Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University, Beijing, China School of Artificial Intelligence, Wuhan University, Wuhan, Hubei, China Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China Suzhou Automotive Research Institute, Tsinghua University, Suzhou, China SynSense Corporation, Ningbo, Zhejiang, China State Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Beijing, China The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Extract — continue reading at the source.
