Brain-inspired Algorithms and Systems

W. Y. Yerima, O. M. Ikechukwu, K. N. Dang and A. Ben Abdallah, "Fault-Tolerant Spiking Neural Network Mapping Algorithm and Architecture to 3D-NoC-Based Neuromorphic Systems," in IEEE Access, vol. 11, pp. 52429-52443, 2023, doi: 10.1109/ACCESS.2023.3278802.
Neuromorphic computing uses spiking neuron network models to solve machine learning problems in a more energy-efficient way when compared to conventional artificial neural networks. However, mapping the various network components to the neuromorphic hardware is not trivial to realize the desired model for an actual simulation. Moreover, neurons and synapses could be affected by noise due to external interference or random actions of other components (i.e., neurons), which eventually lead to unreliable results. This work proposes a fault-tolerant spiking neural network mapping algorithm and architecture to a 3D network-on-chip (NoC)-based neuromorphic system (R-NASH-II) based on a rank and selection mapping mechanism (RSM). The RSM allows the ranking and rapid selection of neurons for fault-tolerant mapping. Evaluation results show that with our proposed mechanism, we could maintain a mapping efficiency of 100% with 20% spare rate and a fault rate (40%) more than in the previous mapping framework. The Monte Carlo simulation evaluation of reliability shows that the RSM mechanism has increased the mean time to failure (MTTF) of the previous mapping technique by 43% on average. Furthermore, the operational availability of the RSM for mapping to a 4×4×4 (smallest) and 6×6×6 (largest) NoC is 88% and 67% respectively.

Abderazek Ben Abdallah, Khanh N. Dang, ''Toward Robust Cognitive 3D Brain-inspired Cross-paradigm System,'' Frontier in Neuroscience 15:690208, doi: 10.3389/fnins.2021.690208
Spiking Neuromorphic systems have been introduced as promising platforms for energy-efficient spiking neural network (SNNs) execution. SNNs incorporate neuronal and synaptic states in addition to the variant time scale into their computational model. Since each neuron in these networks is connected to many others, high bandwidth is required. Moreover, since the spike times are used to encode information in SNN, a precise communication latency is also needed, although SNN is tolerant to the spike delay variation in some limits when it is seen as a whole. The two-dimensional packet-switched network-on-chip was proposed as a solution to provide a scalable interconnect fabric in large-scale spike-based neural networks. The 3D-ICs have also attracted a lot of attention as a potential solution to resolve the interconnect bottleneck. Combining these two emerging technologies provides a new horizon for IC design to satisfy the high requirements of low power and small footprint in emerging AI applications. Moreover, although fault-tolerance is a natural feature of biological systems, integrating many computation and memory units into neuromorphic chips confronts the reliability issue, where a defective part can affect the overall system's performance. This paper presents the design and simulation of R-NASH-a reliable three-dimensional digital neuromorphic system geared explicitly toward the 3D-ICs biological brain's three-dimensional structure, where information in the network is represented by sparse patterns of spike timing and learning is based on the local spike-timing-dependent-plasticity rule. Our platform enables high integration density and small spike delay of spiking networks and features a scalable design. R-NASH is a design based on the Through-Silicon-Via technology, facilitating spiking neural network implementation on clustered neurons based on Network-on-Chip. We provide a memory interface with the host CPU, allowing for online training and inference of spiking neural networks. Moreover, R-NASH supports fault recovery with graceful performance degradation.

Khanh N. Dang, Nguyen Anh Vu Doan, Abderazek Ben Abdallah “MigSpike: A Migration Based Algorithm and Architecture for Scalable Robust Neuromorphic Systems,”  IEEE Transactions on Emerging Topics in Computing (TETC), 12/2021. DOI: 10.1109/TETC.2021.3136028
While conventional hardware neuromorphic systems usually consist of multiple clusters of neurons that communicate via an interconnect infrastructure, scaling up them confronts the reliability issue when faults in the neuron circuits and synaptic weight memories can cause faulty outputs. This work presents a method named MigSpike that allows placing spare neurons for repairing with the support of enhanced migrating methods and the built-in hardware architecture for migrating neurons between nodes (clusters of neurons). MigSpike architecture supports migrating the unmapped neurons from their nodes to suitable ones within the system by creating chains of migrations. Furthermore, a max-flow min-cut adaptation and a genetic algorithm approach are presented to solve the aforementioned problem. The evaluation results show that the proposed methods support recovery up to 100% of spare neurons. While the max-flow min-cut adaption can execute milliseconds, the genetic algorithm can help reduce the migration cost with a graceful degradation on communication cost. With a system of 256 neurons per node and a 20% fault rate, our approach minimizes the migration cost from remapping by 10.19× and 96.13× under Networks-on-Chip of 4×4 (smallest) and 16×16×16 (largest), respectively. The Mean-Time-to-Failure evaluation also shows an approximate 10× of lifetime expectancy by having a 20% spare rate.

 O. M. Ikechukwu, K. N. Dang and A. Ben Abdallah, ''On the Design of a Fault-Tolerant Scalable Three Dimensional NoC-Based Digital Neuromorphic System With On-Chip Learning,'' IEEE Access, vol. 9, pp. 64331-64345, 2021, doi: 10.1109/ACCESS.2021.3071089 
Neuromorphic systems have shown improvements over the years, leveraging Spiking neural networks (SNN) event-driven nature to demonstrate low power consumption. As neuromorphic systems require high integration to form a functional silicon brain-like, moving to 3D integrated circuits (3D-ICs) with three-dimensional network on chip (3D-NoC) interconnect is a suitable approach that allows scalable design, shorter connections, and lower power consumption. However, highly dense neuromorphic systems also encounter the reliability issue where a single point of failure can affect the systems'operation. Because neuromorphic systems rely heavily on spike communication, an interruption or violation in the timing of spike communication can adversely affect the performance and accuracy of a neuromorphic system. This paper presents NASH, a a fault-tolerant 3D-NoC based neuromorphic system that incorporates as processing elements, lightweight spiking neuron processing cores (SNPCs) with spike-timing-dependent-plasticity (STDP) on-chip learning. Each SNPC houses 256 leaky integrate-and-fire (LIF) neurons and 65k synapses. Evaluation results on MNIST classification, using the fault-tolerant shortest-path K-means-based multicast routing algorithm (FTSP-KMCR), show that the NASH system can maintain high accuracy for up to 30% permanent fault in the interconnect with an acceptable area and power overheads when compared to other existing systems.

The H. Vu,Yuichi Okuyama, Abderazek Ben Abdallah, '' Comprehensive Analytic Performance Assessment and K-means based Multicast Routing Algorithms and Architecture for 3D-NoC of Spiking Neurons.,'' ACM Journal on Emerging Technologies in Computing Systems (JETC), Vol. 15, No. 4, Article 34, October 2019. doi: 10.1145/3340963
Spiking neural networks (SNNs) are artificial neural network models that more closely mimic biological neural networks. In addition to neuronal and synaptic state, SNNs incorporate the variant time scale into their computational model. Since each neuron in these networks is connected to thousands of others, high bandwidth is required. Moreover, since the spike times are used to encode information in SNN, very low communication latency is also needed. The 2D-NoC was used as a solution to provide a scalable interconnection fabric in large-scale parallel SNN systems. The 3D-ICs have also attracted a lot of attention as a potential solution to resolve the interconnect bottleneck. The combination of these two emerging technologies provides a new horizon for IC designs to satisfy the high requirements of low power and small footprint in emerging AI applications. In this work, we first present a comprehensive analytical model to analyze the performance of 3D mesh NoC over variants of different SNN topologies and communications protocols. Second, we present an architecture and a low-latency spike routing algorithm, named shortest path K-means based multicast (SP-KMCR), for three-dimensional NoC of spiking neurons (3DNoC-SNN). The proposed system was validated based on an RTL-level implementation, while area/power analysis was performed using 45nm CMOS technology.

The Vu, Ogbodo Mark Ikechukwu, Abderazek Ben Abdallah, ''Fault-tolerant Spike Routing Algorithm and Architecture for Three Dimensional NoC-Based Neuromorphic Systems'', IEEE Access, Vol 7, pp. 90436-90452, 2019, DOI: 10.1109/ACCESS.2019.2925085
Neuromorphic computing systems are an emerging field that takes its inspiration from the biological neural architectures and computations inside the mammalian nervous system. The spiking neural networks (SNNs) mimic real biological neural networks by conveying information through the communication of short pulses between neurons. Since each neuron in these networks is connected to thousands of others, high bandwidth is required. Moreover, since the spike times are used to encode information in SNN, very low communication latency is also necessary. On the other hand, the combination of Two-dimensional Networks-on-Chip (2D-NoC) and Three-dimensional Integrated Circuits (3D-ICs) can provide a scalable interconnection fabric in large-scale parallel SNN systems. Although the SNNs have some intrinsic fault-tolerance properties, they are still susceptible to a significant amount of faults; especially, when we talk about integrating the large-scale SNN models in hardware. Consequently, the need for efficient solutions capable of avoiding any malfunctions or inaccuracies, as well as early fault-tolerance assessment, is becoming increasingly necessary for the design of future large-scale reliable neuromorphic systems. This paper first presents an analytical model to assess the effect of faulty connections on the performance of a 3D-NoC-based spiking neural network under different neural network topologies. Second, we present a fault-tolerant shortest-path k-means-based multicast routing algorithm (FTSP-KMCR) and architecture for spike routing in 3D-NoC of spiking neurons (3DFT-SNN). Evaluation results show that the proposed SP-KMCR algorithm reduces the average latency by 12.2% when compared to the previously proposed algorithm. In addition, the proposed fault-tolerant methodology enables the system to sustain correct traffic communication with a fault rate up to 20%, while only suffering 16.23% longer latency and 5.49% extra area cost when compared to the baseline architectures