Reducing spiking neural network inference latency for behavior cloning with input adaptive leakage

Behavior cloning enables efficient imitation of complex human skills from a limited number of demonstrations. However, artificial neural networks (ANNs), which are commonly employed for policy learning, suffer from their high computational demands and energy consumption. Spiking neural networks (SNNs) offer a promising alternative because of their event-driven and energy-efficient characteristics.

A common training approach for SNNs is to convert the weights from a pretrained ANN into an equivalent SNN. However, due to rate coding, such a converted SNN typically requires multiple inference timesteps for each input to converge firing rates, resulting in increased latency. This issue is especially problematic for continuous sensory inputs such as video frames in robot manipulation.

To address this, we propose an input-adaptive leakage mechanism that dynamically controls the leakage of the membrane potential, an internal state of spiking neurons, according to input variations. When the change between successive inputs is small, the mechanism decreases the leakage to preserve the membrane potential. Conversely, when the input change is large, it increases the leakage to rapidly update the membrane potential with the new input.

We evaluate the proposed method using a hybrid ANN–SNN policy in which only the visual encoder is converted into an SNN. Experiments are conducted on four robotic manipulation tasks in the RoboManipBaselines simulation environment and one real-robot manipulation task. Compared with conventional reset-based SNNs, the proposed method reduces inference timesteps per frame by up to 87.5% and the measured energy of the entire hybrid policy per episode by 22.6% on average.

These benefits diminish when inter-frame variations are large and broadly distributed, in which case the proposed method may require more timesteps and energy. Overall, the proposed method improves the efficiency of conventional SNN-based policies, although ANNs remain more efficient on the evaluated CPU and GPU platforms. This work was funded by JST-ALCA-Next Japan Grant Number JPMJAN24F1.

Department of Communications and Computer Engineering, Graduate School of Informatics, Kyoto University, Kyoto, 606-8501, Japan Department of Information and Communication Engineering, Graduate School of Engineering, Nagoya University, 464-8603, Nagoya, Japan The authors declare no competing interests. Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.