Cognitive Computing with Spin-Based Neural Networks Mrigank Sharad1, Charles Augustine2, Georgios Panagopoulos1, and Kaushik Roy1

1School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana 47907, USA 2Circuit Research Lab, Intel labs, Intel Corporation, Hillsboro, OR, US

msharad@purdue.edu, charles.augustine@intel.com, gpanagop@purdue.edu, kaushik@purdue.edu

ABSTRACT We model a step transfer function neuron with lateral spin valve (LSV) and propose its application in low power neural network hardware. The computational task in such a network is performed by nano-magnets, metal channels and programmable conductive elements, that constitute the neuron-synapse units and operate at a terminal voltage of ~20 mV. CMOS transistors provide peripheral support in the form of clocking, power gating and inter-neuron signaling. Simulations for cognitive as well as Boolean computation applications show more than 94% improvement in power consumption as compared to a conventional CMOS design at the same technology node.

Categories and Subject Descriptors B.7.1 [Integrated Circuits] Types & Design Styles - Advanced Technologies General Terms Performance, Design

Keywords Neural network, spin valve, low power design, magnets

1. INTRODUCTION Recent experiments on lateral spin valves (LSV) have demonstrated switching of nano-magnets with non local spin transfer torque (STT) [1, 2]. It involves generation of spin potential difference across a magnet-metal interface using spin polarized current flow in the metal, and, results in flipping of the magnet with pure spin current, without direct charge current injection into it [3, 4]. The metallic device structure of an LSV allows application of very small terminal voltages, resulting in low switching energy for the magnets. Owing to analog nature of spin current, an LSV can perform non-Boolean computation. Application of LSV as a majority gate has been proposed earlier [5]. In this work we propose a device model for neuron based on LSV. We show that, with appropriate clocking scheme, an LSV with complementary polarizer inputs acts as a neuron. The proposed neuron device can be integrated in a large-scale crossbar-network architecture, with programmable or fixed conductive elements as synapses. We also propose a model for cellular neuron with domain wall magnets (DWM) [6] as programmable synapses. It can be suitable for low power, cellular neural network (CNN) hardware which employs neighborhood connectivity. The compact, low resistance, magneto-metallic neuron units allow synapse current flow across a small terminal voltage Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. DAC 2012, June 3-7, 2012, San Francisco, California, USA. Copyright 2012 ACM 978-1-4503-1199-1/12/06...$10.00

of ~20 mV, resulting in low computation power. In this paper we briefly describe the system level integration scheme for the proposed neuron model. Rest of the paper is organized as follows. We introduce the proposed device structures for spintronic neuron in section 2 System level information is briefly discussed in section 3. Section 4 presents the simulation framework, synthesis and simulation results for the proposed design scheme. Finally, section 5 concludes the paper.

2. SPIN BASED NEURON-SYNAPSE UNITS Device structure for the neuron model is shown in fig. 1a. The input magnets m1 and m2 act as complementary spin polarizers, and, receive input current from positive and negative weight synapses, that constitute of via’s or programmable conductive elements like memristor or phase change memory (PCM) [7]. The neuron magnet m4, forms the free layer of a magnetic tunneling junction (MTJ). The third input magnet m3 is used to realize current-mode Bennett clocking for the neuron magnet m4

Fig. 1 (a) Neuron with via-synapse (b) neuron switching waveform

Current injected through m3 into the channel, pushes the neuron magnet to the unstable hard axis state (fig. 1b). After removal of the hard axis bias, the neuron magnet makes a fast transition to one of the stable states, governed by the net spin polarity of the charge current injected into the channel. Thus m4 in effect compares the amount of current received from the positive and the negative weight synapses, connected to m1 and m2 respectively. Owing to low stability of the hard axis state, even a small difference in the currents received by m1 and m2 effects the switching. Hence, in effect, a zero threshold, bipolar step function of a neuron is realized. The ‘near’ zero spin current threshold for the neuron also reduces the required current injection through input synapses to few micro amperes. The proposed spintronic neuron can be integrated with large number of programmable or non-programmable ‘conductive’ synapses to arrive at low power computational networks. Programmable ‘spintronic’ synapses can be realized using domain wall magnets (DWM). For the lateral DWM-channel interface shown in fig. 2b, the spin polarity of the current injected into the

1262

channel through the DWM depends upon the DW location (fig. 2a). The DW can be programmed by a lateral injection of charge current through the DWM [6]. Hence, a DWM can acts as a low resistance, compact and programmable synapse, leading to homogenous and modular spin-mode neuron-synapse unit (fig. 2c). The limited spin diffusion length of metal channel however restricts the number of DWM synapses per neuron [3], and hence this structure is more suitable for CNN architecture with neighborhood connectivity. Both the neuron models, with conductive and spintronic synapses respectively, can compute with analog as well as binary inputs. This is because, the device structure involves current mode operation, that is analog in nature.

Fig. 2(a) spin polarization as a function of DW location (b)DWM synapse with channel interface (c) cellular neuron with programmable domain wall synapse

3. SYSTEM INTEGRATION Since spin signals can not be transmitted over a distance longer than a few spin diffusion length of the metal channel [3, 4], we employ charge mode signaling to interconnect the spintronic neurons. A dynamic CMOS latch (fig. 3a) senses the state of the neuron MTJ and drives the current source transistor, which transmits synapse current to all the fan-out neurons. Note that, the synapse input currents, involved in computation, flow across a small terminal voltage V (fig. 3b), thereby reducing the static power consumption resulting from large number of synaptic communications.

Fig. 3(a) CMOS detection unit senses the state of the neuron magnet and (b) transmits synpse current to the fanout neurons across a small terminal voltage V. The aforemensioned detection scheme can be applied to both the neuron strcutures described in the previous section. As mensioned before, the DWM based neuron is suitable for a CNN design (fig. 4a) whereas the four termial neuron in fig. 1a is a suitable candidate for a network with programmable or fixed conductive crossbar elements (fig.4b)

4. SIMULATION AND PERFORMANCE Simulation of the spin device structures presented in this work, involves self consistent solution of the spin transport using Valet Fert’s spin diffusion model and the magnet dynamics using Landau-Lifshitz-Gilbert equation LLG [3]. This simulation

framework has been benchmarked with experimental results for lateral spin valves.

Fig. 4(a) CNN architecture with neighborhood connectivity (b) Fully connected crossbar network architecture

Neural networks can perform both Boolean as well as cognitive computations (fig. 5). The network weights are obtained by offline training. For arithmetic computation blocks, like multipliers and adders, the required network size grows exponentially beyond an input dimension of 4x4, because of large training set. Hence, for larger number of input bits, the overall computation is decomposed into 3x3 or 4x4 units in order to obtain maximum benefits. Fig. 5c compares the performance of the spin-CMOS (45nm) hybrid network with 45nm-CMOS design for some benchmark cognitive and Boolean applications.

Fig. 5 (a) Edge detection from grey scale image using DWM based CNN (b) Charater recognition with DWM based network (c) Power consumtion of proposed desgin scheme relative to 45nm CMOS at 500 MHz frequecy.

5. CONCLUSION We employed spin device phenomena like, majority evaluation and hard axis switching to model compact and low energy neuron-synapse units. The proposed circuit/architecture can lead to ultra low power spin-CMOS hybrid neural networks for cognitive as well as Boolean computations that can achieve more than 94% lower power as compared to a CMOS design. ACKNOWLEDGEMENT: This research was funded in part by Nano Research Initiative and by the INDEX center

REFERENCES [1] Kimura et. al, Phys. Rev. Lett. 2006 [2] Sun et. al., Appl. Phys. Lett. 2009. [3] Behin-Ain et. al., Nature Nano. 2010 [4] Behin-Ain et. al, Appl.Phys.Lett. 2011 [5]. Augustine et al., Nanoarch, 2011 [6] M. Sharad et al., IJCNN, 2012 [7]Kuzum et. al., Nano Lett., 2011

1263