Composing neural algorithms with Fugu

Composing neural algorithms with FuguJames B. Aimone*, William Severa, Craig M. VineyardCenter for Computing Research, Sandia National Laboratories

P.O. Box 5800Albuquerque, New Mexico 87185-1327

*[email protected]

ABSTRACTNeuromorphic hardware architectures represent a growing fam-ily of potential post-Moore’s Law Era platforms. Largely due toevent-driving processing inspired by the human brain, these com-puter platforms can o�er signi�cant energy bene�ts comparedto traditional von Neumann processors. Unfortunately there stillremains considerable di�culty in successfully programming, con-�guring and deploying neuromorphic systems. We present theFugu framework as an answer to this need. Rather than necessi-tating a developer a�ain intricate knowledge of how to programand exploit spiking neural dynamics to utilize the potential bene�tsof neuromorphic computing, Fugu is designed to provide a higherlevel abstraction as a hardware-independent mechanism for link-ing a variety of scalable spiking neural algorithms from a varietyof sources. Individual kernels linked together provide sophisti-cated processing through compositionality. Fugu is intended to besuitable for a wide-range of neuromorphic applications, includingmachine learning, scienti�c computing, and more brain-inspiredneural algorithms. Ultimately, we hope the community adopts thisand other open standardization a�empts allowing for free exchangeand easy implementations of the ever-growing list of spiking neuralalgorithms.

ACM Reference format:James B. Aimone*, William Severa, Craig M. Vineyard. 2016. Composingneural algorithms with Fugu. In Proceedings of , , , 8 pages.DOI: 10.1145/nnnnnnn.nnnnnnn

1 INTRODUCTION�e success of neuromorphic computing technologies is dependenton its large-scale adoption as a post-Moore’s law, low power solu-tion for multiple applications. �e generality of neural computingapproaches is an area of active exploration [1, 2], but there is alreadya growing disparity between this value of neuromorphic systemsas a general tool and emerging so�ware stacks for leveraging theseplatforms for speci�c functions. For individual applications, suchas spiking deep neural networks (DNNs), pre-de�ned specializedsolutions are o�en su�cient. An example of this is the Whetstoneso�ware, which we recently introduced [11] as a tool to convertstandard Keras-trained DNNs to a spiking-compatible structure.Similarly, there are an increasing number of options for neural pro-gramming environments that are premised on established classes ofneural computation. Tools such as PyNN [4], Nengo [3], and N2A[8] assume certain intent among user communities, and thus while

,2016. 978-x-xxxx-xxxx-x/YY/MM. . .$15.00DOI: 10.1145/nnnnnnn.nnnnnnn

o�en powerful, these implicitly require that users orient themselvesto a certain perspective of neural computation.

Here, we sought to develop a programming platform to enablethe development of neuromorphic applications without substan-tial knowledge of neural computing or neuromorphic hardware.Our solution, which we refer to as Fugu1, is intended to facilitateneuromorphic application development in a manner similar to howCUDA facilitates the programming of GPUs.

Fugu is structured so as to separate the task of programmingapplications that may leverage neuromorphic hardware from thedesign of spiking neural algorithms (SNAs) and the speci�c detailsof neuromorphic hardware platforms. We accordingly foresee threecategories of users. �e primary target population is the generalcomputer programming community; well-versed in scienti�c com-puting approaches but perhaps only modestly familiar with parallelcomputing and likely unfamiliar with neural approaches. For theadoption of neuromorphic hardware it is critical that these userscan leverage this technology. To enable this, a second group ofusers - those capable of designing SNAs - also need to be able toincorporate their algorithms into the Fugu construct. �is popula-tion of users may be well versed in neural computation generally,but also may not be familiar with speci�c considerations of dif-ferent neural hardware platforms. Finally, the third category ofusers would be those who are deeply familiar with neuromorphichardware, and are capable of optimizing and tailoring generic SNAsinto algorithmic implementations that are optimized to the currentconditions of neuromorphic hardware.

2 BACKGROUNDFugu is a high-level programming framework speci�cally designedfor develolping spiking algorithms in terms of computation graphs.At the lowest level, SNAs are directed graphs, with nodes corre-sponding to neurons and edges corresponding to synapses. How-ever, by considering how SNAs can be networked together (notunlike how the brain consists of networks of local circuits), Fugu isable to consider a higher level graphical perspective of the overar-ching computation.

A key underpinning of Fugu is that SNAs, if properly constructed,should be composable. In the context here, this means that twoappropriately constructed SNAs can be combined to form largeralgorithms in that the outputs of one can be used as the inputs tothe other. �e following sections will highlight how this conceptof compositionality is more than simply matching sizes. For two

1�e name Fugu is inspired by the Japanese word for pu�er�sh; which, of course, havespikes. Furthermore, Fugu is considered a culinary delicacy due to the presence of lowlevels of the neurotoxin tetrodotoxin, or TTX, which has signi�cant value in studyingthe electrophysiology mechanisms underlying biological action potentials. Only oneauthor (CMV) has eaten Fugu before.

arX

iv:1

905.

1213

0v1

[cs

.NE

] 2

8 M

ay 2

019

, , James B. Aimone*, William Severa, Craig M. Vineyard

small SNAs to be combined into a larger SNA, the sizes, temporalstructure, and encoding schemes have to align. �is process canbecome non-trivial when a number of SNAs are to be composedtogether, the automation of which is the primary contribution ofFugu.

Fugu is designed primarily for spiking algorithms, the core modelfor which is described in section 2.1 below. However, this emphasison spiking is not meant to preclude its future use for architectureswhich may achieve bene�ts from other features such as analogarrays. Most neural algorithms, whether spiking or not, can beconsidered through a graphical perspective and thusmay be suitablefor the approach described here.

2.1 Generic spiking neural algorithm modelFugu assumes a fairly simple neural model for SNAs, so as to enablethe generic use of spiking neuromorphic hardware. �e default neu-ron model of Fugu is that of leaky-integrate and �re (LIF) neurons,with parameterizable voltage thresholds (Vthresh), time constantsτi , and reset voltages Vreset. �e default synapses are consideredpoint synapses, with a parameterized weight wi, j , where i and jare source and target neurons respectively. At each timestep, eachneuron computes the voltage at time t given by Vt in the followingmanner

Vj (t) =∑i

fi (t) ∗wi, j +Vj (t − 1),

Vj =

{Vreset if Vj > Vthresh(1 − τj )Vj elsewise.

,

fj (t) ={P if Vj > Vthresh0 elsewise.

where the �ring, fj (t) is determined if the neuron’s voltagecrosses a threshold. To account for probabilistic neural transmis-sion, P is a probabilistic Bernoulli draw of either 0 or 1 accordingto a stochastic �ring rate at rate p. If neurons are not stochastic,p = 1.

2.2 Learning, structural plasticity, and otherextensions to core spiking model

�e design of Fugu allows for capabilities beyond this basic model.For instance, if a given algorithm requires a learning mechanismwith a particular learning coe�cient, that can be included as ana�ribute of speci�c synapses, but not all neuromorphic hardwaremay be capable of implementing that algorithm correctly. Similarly,more sophisticated neuron models, such as multi-compartmentmodels (which leverage dendritic dynamics) and conductance-basedmodels are entirely valid from an algorithm design perspective,however there are few hardware platforms that can fully leveragethese.

�ese added complexities will likely arise with more sophisti-cated algorithms, particularly from biologically-inspired models.Fugu thus remains somewhat agnostic to what occurs beyond thebasic LIF model within a given SNA, although such functions maypresent a risk that the algorithm may not be compatible with adownstream platform. Accordingly, one key requirement is thatthe communication and connections between algorithms - which is

the operational domain of Fugu, is expressly compatible with thestandard LIF model. In this sense, component SNAs must presentdiscrete spike events as outputs and similarly (with some excep-tions) take in spike events as inputs.

3 DESIGN OF FUGUAs stated above, a key goal of Fugu is to provide a general scien-ti�c computing user access to emerging neuromorphic hardware— speci�cally spiking neuromorphic hardware — by providing anaccessible library of functions that Fugu can map into neural hard-ware. Figure 1 provides an overview of the Fugu framework whichwe will elaborate upon the subcomponents in the following sec-tions.

Fugu accomplishes this by providing the following core capabili-ties:

• AnAPI to conventional programming environments(i.e., Python, C++)• Automated construction of a graphical intermediate

representation of spiking neural algorithms• Outputs to neural hardware compilers or Fugu’s ref-

erence simulator

�e following sections expand on each of these topics.

3.1 API�e goal of the API for Fugu is to make the construction of a Fugualgorithm be readily called from C++ or Python. In theory, all ofthe SNA requirements and processing should be transparent to auser; such that they only have to call a function with standard I/Oprotocols; not unlike a hardware interface language such as CUDA.

�e intent of Fugu is to enable the user to program an applicationto neuromorphic hardware with only limited knowledge of theunderlying architecture or requisite algorithms. �us, the API level,which is the interface that we expect to be most commonly used,simply requires the user to de�ne a computational graph, whichwe call a sca�old. Each Fugu sca�old consists of nodes, known asbricks, which are the component SNAs, and edges between thosebricks that de�ne the �ow of information.

During the design of a Fugu application, a user would constructa computational graph of an application. Consider a simple applica-tion with four operations: function A processes an input, functionsB and C each process the output of A, and function D combinesthe outputs of B and C. As shown in the pseudocode within Fig-ure 2, from a simple set of instructions of how to connect thesefour functions, Fugu would construct bricks for each function andcompose them into a larger algorithm sca�old.

�is sca�old is a graphical representation of the desired Fugualgorithm, however this perspective remains agnostic the eventualpopulations of neurons that will perform the desired computation.Each of the bricks consists of instructions for building its particularneural algorithm, which can take any number of forms. To becomefunctional, the overall sca�old must progress from individual brickrepresentations which must be populated with the appropriateinternal neural circuits that have been appropriately scaled andcon�gured to interface with all of the brick’s neighbors.

Composing neural algorithms with Fugu , ,

Figure 1: An overview diagram of a common work�ow in Fugu (Green squares). Code exists to underlie this work�ow (Bluehexagons) with examples provides (Tan diamonds).

3.2 Intermediate Representation�e primary contribution of Fugu is the managed and automatedIR between higher-level coding environments and low-level neu-romorphic hardware and their compilers. �is IR consists of threecomponents that, during compilation, provide the connection be-tween the API and the compiler output: a library of SNA bricks, acollection of algorithms for linking SNA bricks, and the combinedapplication graph output.

�e IR of Fugu exists within Python, and it leverages the Net-workX libary [5] to construct and manage the neural circuits thatwill be generated and combined during Fugu operation.

3.2.1 Library of SNA Bricks. �e Fugu library consists of a grow-ing set of SNA bricks that are suitable for being composed togetherinto larger functional units. Importantly, the algorithm bricks thatare contained within Fugu generally do not consist of explicit neural

circuits, but rather they are scripts that can generate the appropri-ate neural circuit for a given application. �is allows them to besized appropriately to tailor them to interface with predecessors.

For example, we consider the constant-time 1-dimension maxcross-correlation algorithm in [10]. �at algorithm compares twobinary vectors of length N by having a dedicated neuron withinan intermediate layer calculate each potential o�-set, requiring anintermediate layer of size N 2. Subsequently, an output layer, sized2N − 1 samples that intermediate layer to determine the relativeshi� between inputs that yields the maximal overlap. Figure 3illustrates these two di�erent SNAs with their di�ering resourcerequirements.

As the description shows, the exact neural circuit needed ishighly speci�c to N . In addition, the resulting circuits have otherimportant properties that will be necessary at later Fugu steps. Forinstance, for the constant-time algorithm, the inputs are provided


Figure 2: Fugu Sca�old and Bricks

Figure 3: Two SNAs for computing cross-correlation requr-ing di�erent space and time complexities

all at the same time (Tin = 1), and the output is always a single time-step (Tout = 1) arriving two time steps later (D = 2). An alternativeversion of this algorithm streams the inputs in over time and usesdelays to make the same computation with fewer neurons, albeit atan extended time cost, a characteristic that will produce di�erentneural circuit and associated metadata. It is also important to notethat this metadata may be a function of input parameters as well —for instance in the matrix-multiplication application described in[7], there is a version of the algorithm whose depth isO(log logN ),where N is the size of the largest matrix dimension.

A schematic of a Fugu brick is shown in Figure 4. Some propertiescommon for bricks within Fugu are:

• Nin – number of input neurons

Figure 4: Spiking Algorithm as Fugu Brick

• Tin – time length of inputs (how long do inputs stream in).=inf if streaming

• Nout – number of output neurons• Tout – time length of output. = inf if streaming• D – circuit depth, corresponding to how many global

timesteps must pass for the input at t=1 to reachHowever, ultimately the determination of the required parame-

ters is dependent on the coding scheme. To abstract these concepts,we incorporate a local ‘index’ to each of a brick’s output neurons. Aneuron’s index indicates (to any downstream brick) the informationrepresented by that neuron (e.g., which component of a coding isrepresented by that neuron). By adopting a standardization on thisindexing, bricks are able to communicate with one another with-out imposing design considerations within the bricks themselves.Additionally, the index provides a �exible mechanism supportingan arbitrary number of dimensions or subsets.

To exist within Fugu, each of the bricks must be able to take in-structions from the Fugu environment and produce the appropriateneural circuit NetworkX graph per the necessary scales and timings.�ese circuits are referred to as local circuits, and are standalonecircuit SNAs for computing the requisite function.

3.2.2 Linking code to combine local circuits into global circuit.Once a sca�old is constructed by linking bricks together, the sca�oldmust build a comprehensive global graph (which we call a circuit).�is circuit is a platform-independent intermediate representationthat becomes complied down to platform-speci�c code. Fugu buildsthe circuit using a lazy method, iterating over the nodes of thesca�old graph. When a brick is capable of building itself (due tothe determination of parameters upstream), it builds its local graphaccording to the build-time requirements of the sca�old and thislocal graph is incorporated into the global circuit. �e process oflaying bricks is seamless and automatic to the end user, and brickdevelopers only need to manage their own local graphs.

�ere are two primary challenges and a number of routine stepsthat require care for linking these circuits together.

Align sizes of bricks. Eachmodel has an input sizeNin and an out-put size Nout , and in order for two bricks to be compatible with oneanother serially, then it is necessary that the downstream module isscaled appropriately to generate a graph suitably sized to take theoutputs. n general, the shape of the input determines the remainderof the graph. So, when a user de�nes the input shape (via inputbricks), Fugu can determine the required shape, dimensionality, orother parameters of any connected bricks automatically.

A general scheme is shown in Figure 5.


Figure 5: Normalizing brick sizes

Align timings of bricks. Each of the bricks requires some amountof computation time or ‘circuit depth’. If bricks are intended to runin parallel, the di�erence in timings and depths may require thatan additional delay circuit is instantiated on one of the pathways toensure that the circuit is appropriately timed. Figure 6 illustratesthis idea.

As the branches may be indexed arbitrarily and the time or depthof a module may be undetermined until its overall size is identi�ed,it is unknown at the start of Fugu which branch will be the limitingfactor in terms of time. �is longest branch is the reference lengththat all other branches must match. Once this branch depth isfound, we then work through each of the other branches to makethe depths equivalent.

If the branch depth can be fully determined at build time, we cansimply add a delay block - most simply a set of repeater neuronsthat spike with a delay of whatever the di�erence is. Most simply,this delay block could be at the end of the branch. However, thereis likely a bene�t to load balance over time; the delays will berelatively cheap in terms of computation, and thus they can perhapsbe staggered at di�erent times of each branch to keep the overallnetwork activity at a roughly comparable level.

If the branch depth is variable or dependent on the input data (e.g.an iterative algorithm with depth determined at runtime), then wecan stage information other branches in a bu�er until all brancheshave completed their computation. �is bu�er can then be releasedusing control nodes—Extra neurons instantiated in the graph tosignal control commands. Each brick de�nes at least one controlnode that �res on completion. �is signal can then be used to �usha bu�er and continue the computation.

�ere is also a third potential solution, though it would requiremore careful implementation. Many of the SNAs being developedcan be tailored to use fewer neurons if more time is available. �istime-space tradeo� is generally biased towards the faster algorithm;however in cases where a module with such a tradeo� sits withina branch with “free-time” so to speak, it is possible, and perhapseven advantageous, to represent that module in a more time-costly,space-e�cient manner that reduces the overall neuron footprint ofthe model.

3.2.3 Output. �e output of Fugu will be a single NetworkXgraph that fully describes the spiking neural algorithm. �e edgesof this graph will be the synapses of the model, and will accordingly

Figure 6: Adding a delay to synchronize Fugu branches

have weights associated with them as a�ributes. �e nodes of thisgraph will be the neurons of the model, and will accordingly havedynamics parameters associated with them. Additionally, someparameters, such as learning rates, additional dynamical states, etc.may be included within the a�ributes.

3.3 Neuromorphic Hardware Compilers, theReference Simulator, and IntentionalLimitations of Fugu

�e primary output of Fugu is the above-stated NetworkX graphthat represents the desired neural algorithm, but is also hardwareagnostic. Currently, each major neuromorphic platform has its ownunique programming interface, and thus it is reasonable to assumethat for the near future the need for a “neuromorphic hardwarecompiler” will be met by one-o� solutions speci�c to di�erentplatforms.

Because of this, we envision that as these hardware-speci�cinterfaces begin to stabilize, it will hopefully be straightforwardto convert from this generic NetworkX computational graph de-scription of the Fugu algorithm to any required hardware format.However, given that these platforms remain a �uid target, Fugualso includes a reference simulator which enables algorithms to beprototyped.

�e reference simulator is designed to be conservative: clearlythe breadth of what simulations can implement far outstrip currentneuromorphic hardware capabilities today; especially when dynam-ics such as synaptic plasticity and architectural recon�guration areconsidered. As such, because the Fugu simulator is intended toshow that an implemented algorithm would be capable of runningon a generic neuromorphic system, we do not yet include learn-ing, complex neuron types, or other capabilities beyond the basicLIF model. �is is not a judgment on the value or importance ofthose models; but rather a re�ection of the state of the hardwarecommunity. As these platforms capabilities universally begin tomove to include these dynamics, the reference simulator will ad-vance accordingly to encompass these capabilities. Furthermore,the reference simulator is not intended to provide indications of theruntimes and other performance measures neuromorphic hardwarecan enable.


Figure 7: Combined Bricks

Meanwhile, Fugu bricks can be constructed that include node(neuron) and edge (synapse) a�ributes that relate to learning rulesor other extended dynamics, but these would not be assured tooperate as expected on all neuromorphic platforms.

4 EXAMPLESAs follows are four illustrative examples of how bricks and sca�oldsmay be de�ned in Fugu to compute a suite of computations rangingfrom basic logic operations to higher level applications such asgraph analytics or scienti�c computing. In these examples, di�erentprinciples of the spiking dynamics are utilized to infer the result ofthe computation. �is may include the times when spikes occur aswell as the speci�c neurons which spike.

4.1 LogicAs a simple example brick, we �rst present the logical AND func-tion. �is canonical logic operation outputs true (or one) only whenall of its inputs are simultaneously true. A neuron is able to com-pute this operation when nominal inputs each contribute to thereceiving neuron whose threshold is equal to the number of inputs.A leakage value causing the neuron to return to a resting potentialof zero every time-step resets the neuron requiring all inputs mustbe received simultaneously. While simplistic, this intuitive brickexample illustrates the de�nition of neuron parameters and can becomposed in more elaborate sca�olds capable of executing Booleanlogic functions.

Additionally, this brick is easily modi�ed to perform a logicalOR. You simply lower the threshold value so that any incomingspike is enough to have the OR neuron �re. In this case, as with theAND neuron, the boolean function is computed exactly and can beused across coding schemes as long as the shapes of the incomingbricks are equal.

In general, we do not expect e�ciency gains (either wall-timeor theoretical) from implementing boolean logic on neuromorphicsystems. However, these basic components (alongwith other simplefunctions) are key to creating larger, more sophisticated data �owsall while staying on-hardware, avoiding costly I/O with a hostsystem.

4.2 Graph AnalyticsGiven a graph, a core computation is determining the distance be-tween two nodes. We can easily instantiate this as a Fugu brick bytaking a target graph (i.e., the graph on which we are determin-ing distances) and creating a spiking neural network where eachnode is represented by a neuron and each graph edge is a synapse.For a un-directed graph, we create a synapse in both directions.For a weighted graph, the weight is converted to a proportionalsynaptic delay. �en, we simply set the parameters such that (1) aneuron �res when receiving any spike and (2) a neuron only �resonce (this can be done by a very low reset voltage or by a self-linkwith strong negative weight). �en, we begin the algorithm forc-ing the source node/neuron to spike. �e shortest path distancebetween the source neuron and any other neuron is computed bythe time it takes for the target neuron to spike. �is algorithmhas been described independently several times, including [6], andour inclusion here is not novel. However, we do now remark howsuch an algorithm could �t within a larger Fugu sca�old for morecomplicated algorithms.

First, we note that the algorithm is ‘started’ by having the sourcenode �re. Fugu can provide an input with equivalent dimensionalityas the nodes on the graph, essentially a rasterized image of thetarget graph in neurons. �is means that we can use Fugu-stylepreproccesing functions to determine the source neuron. A simpleexample of suchwould be using an image classi�er to identify pointsof interest in overhead imagery. �e user can then cross-referencethat overhead imagery with a map of the area, and Fugu e�ectivelycomputes distances on that map according to the location of objectsin the imagery.

Second, the output of the path-length determination is tempo-rally coded—�e amount of time it takes for the target neuron tospike is equal to the length of the shortest path from the sourcenode to the target node. �is temporal representation is one of thecore number schemes supported in Fugu. As such, we can thentake the output of this brick and perform in-Fugu post-processing.For example, we can easily compare the value against a referencevalue using a �reshold brick (e.g., computing if one has enoughfuel to make a trip) or compare against other path lengths in a


�rst-come-�rst-serve Minimum brick (e.g., determining who wouldget to a destination �rst).

�ese handful of example extensions only being to describe thetypes of complex algorithms possible through compositionality.Despite each of the above bricks being simple in there own right,they can combine together for sophisticated calculations.

4.3 Game�eoryBuilding upon the prior two examples, cast to a temporal regime,a neural computation graph can be used to determine the purestrategy Nash equilibra of some game theoretic problems. Forexample, consider the canonical two player prisoner’s dilemmagame. In this scenario, two captured prisoners are interrogatedseparately and the prison sentences they receive are dictated bytheir decisions to confess and testify against their partner or to notconfess and stand �rm in their allegiance to their criminal partner.Each criminal has preferences over their sentencing, namely toreceive a shorter sentence, and the game dynamics are de�ned suchthat what is best for an individual prisoner is not the same as whatis best for the two criminals jointly.

Representing all pairs of action selections as neurons, and theplayer preferences as the distances between a player neuron and theaction space neurons, a graph for determining pure strategy Nashequilibria may be built. �e Nash equilibria is a solution conceptidentifying outcomes in which no single player can improve uponthe result by changing their own strategy. To identify these equi-librium points, an action neuron can only �re if it receives a spikefrom both players. �is may be achieved either by constructing asca�old using AND bricks such that both inputs must be receivedsimultaneously, or by using the graph analytic distance betweennode dynamics to incorporate a �ring threshold of two and fullleakage every timestep. �e later approach essentially embeds alogical AND function as the neuron activation function with theconnectivity structure of the graph imposing that the inputs comefrom separate input source nodes.

In this sense, while individual players may prefer other outcomesand e�ectively send spikes to their closer preferred actions soonerin time, it is only the joint activation of player actions which drivea neuron to �re. And e�ectively, the determination of the Nashequilbrium is computed by the spiking of action pair neurons. �ispremise may be extended to include more actions as well as largeraction spaces, however for clarity of explanation we described thesimple two player two action game here.

4.4 Scienti�c ComputingOur �nal example is an illustration of how a standalone neuralalgorithm, in this case targeting scienti�c computing, can be brokendown into separate bricks that potential could have alternative usesin contributing to other algorithms. Previously, we described twoneural algorithms for simulating a Markov random walk processto approximate the di�usion equation [9]. One of the algorithms,termed the particle method in our previous study, simulated randomwalk particles directly by dedicating a sub-population of neurons tosimulate each walker. A solution to the di�usion equation (and inturn many other partial di�erential equations for which di�usion is

Figure 8: Schematic of Grid Random Walk Implementationin Fugu. Each block corresponds to a distinct circuit gener-ated by Fugu which are then composed together to simulatethe random walk process.

a key kernel) can then be achieved by taking population statisticsof the random walkers at di�erent time-steps.

�e random walk particle model conceived in [9] is a mono-lithic neural circuit consisting of many walkers that each consist ofseveral rings of neurons and control neurons that implement therandom movements. In e�ect, this model combines two functionsthat are repeated for each walker - random movement neurons anda population that tracks position.

�e Fugu implementation, as summarized in Figure 8 breaksthis monolithic neural circuit into three separate components. Forsimplicity, what is illustrated is a grid torus that tracks positionover a hexagonal grid, inspired by the grid cells in the brain and therecent �nding that biological navigation leverages random walks aswell [12, 13]. In Fugu, we separate the random walkers into threeparts: a binary random number generator brick, a binary tounary conversion brick, and a grid position tracker brick, thela�er of which will be repeated at several di�erent sizes for eachrandom walker that is moving over two dimensions.

Notably, by breaking the random walk algorithm into thesecomponents, future design decisions rapidly become evident. De-pending on the application needs and algorithm / hardware e�-ciency, a �nal brick could be designed that implements the ChineseRemainder �eorem to convert from the grid modular code to ahistogram of random walker locations over space. Alternatively,a hippocampus-inspired place cell brick could be designed thatperform this conversion approximately.

5 DISCUSSION & CONCLUSIONS�e Fugu framework, as introduced here, provides a powerful abil-ity to abstract the underlying neural dynamics which emergingneuromorphic hardware is striving to enable. Not only does thisabstraction and composition capability enable a means by which


non-experts can consider the bene�ts of SNAs in their compu-tations, but it also helps to enable the exploration of when it isadvantageous to do so. In some cases, it has been shown that a SNAapproach is only theoretically bene�cial for problems of speci�cscale. Fugu enables the composition of SNAs such that individ-ually their bene�ts may be limited, but the resultant compositecomputation may be favorable factoring in full circuit costs. Forexample, Figure 7 shows the combination of bricks for Strassenmatrix multiplication and SpikeSort which together may yield thecomputation of a sorted multiplication.

Leveraging the analysis of algorithm composition Fugu enables,it may also help provide insight into the impact of architecturespeci�c design choices when mapping to speci�c hardware plat-forms. For example, if architecture limitations necessitate extensiveuse of repeater or delay circuitry that exceeds either the availableresources of the architecture or drive up the runtime of the compu-tation these insights may suggest another hardware be consideredor alternative algorithmic formulation to meet performance con-siderations. Alternatively, it is also possible that by leveraging theFugu provided IR, neuromorphic compilers may explore additionaloptimization of the circuit layout which the speci�c architecturalfeatures of a given neuromorphic platform may be able to uniquelyenable.

By providing Fugu as a framework for developing and compos-ing spiking algorithms in terms of computation graphs we striveto further the development of SNAs, enable non-experts to utilizeneuromorphic computing in a variety of domains such as scien-ti�c computing, and iterate with neuromorphic architecture devel-opment to jointly understand the impact of design choices bothalgorithmically and architecturally.

ACKNOWLEDGMENT�is research was funded by the Laboratory Directed Researchand Development program at Sandia National Laboratories andthe DOE Advanced Simulation and Computing program. SandiaNational Laboratories is a multi-mission laboratory managed andoperated by National Technology and Engineering Solutions of San-dia, LLC., a wholly owned subsidiary of Honeywell International,Inc., for the U.S. Department of Energy’s National Nuclear SecurityAdministration under contract DE-NA0003525.

�is article describes objective technical results and analysis.Any subjective views or opinions that might be expressed in thepaper do not necessarily re�ect the views of the US Department ofEnergy or the US Government.

REFERENCES[1] James B Aimone. 2019. Neural algorithms and computing beyond Moore’s law.

Commun. ACM 62, 4 (2019), 110–119.[2] James B Aimone, Kathleen E Hamilton, Susan Mniszewski, Leah Reeder, Cather-

ine D Schuman, and William M Severa. 2018. Non-Neural Network Applicationsfor Spiking Neuromorphic Hardware. In Proceedings of the �ird InternationalWorkshop on Post Moores Era Supercomputing. 24–26.

[3] Trevor Bekolay, James Bergstra, Eric Hunsberger, Travis DeWolf, Terrence CStewart, Daniel Rasmussen, Xuan Choo, Aaron Voelker, and Chris Eliasmith.2014. Nengo: a Python tool for building large-scale functional brain models.Frontiers in neuroinformatics 7 (2014), 48.

[4] Andrew P Davison, Daniel Bruderle, JochenM Eppler, Jens Kremkow, Eilif Muller,Dejan Pecevski, Laurent Perrinet, and Pierre Yger. 2009. PyNN: a commoninterface for neuronal network simulators. Frontiers in neuroinformatics 2 (2009),11.

[5] Aric Hagberg, Pieter Swart, and Daniel S Chult. 2008. Exploring network structure,dynamics, and function using NetworkX. Technical Report. Los Alamos NationalLab.(LANL), Los Alamos, NM (United States).

[6] Kathleen E Hamilton, Ti�any M Mintz, and Catherine D Schuman. 2019. Spike-based primitives for graph algorithms. arXiv preprint arXiv:1903.10574 (2019).

[7] Ojas Parekh, Cynthia A Phillips, Conrad D James, and James B Aimone. 2018.Constant-Depth and Subcubic-Size �reshold Circuits for Matrix Multiplica-tion. In Proceedings of the 30th on Symposium on Parallelism in Algorithms andArchitectures. ACM, 67–76.

[8] Fredrick Rothganger, Christina EWarrender, Derek Trumbo, and James BAimone.2014. N2A: a computational tool for modeling from neurons to algorithms.Frontiers in neural circuits 8 (2014), 1.

[9] William Severa, Rich Lehoucq, Ojas Parekh, and James B Aimone. 2018. Spikingneural algorithms for markov process random walk. In 2018 International JointConference on Neural Networks (IJCNN). IEEE, 1–8.

[10] William Severa, Ojas Parekh, Kristofor D Carlson, Conrad D James, and James BAimone. 2016. Spiking network algorithms for scienti�c computing. In RebootingComputing (ICRC), IEEE International Conference on. IEEE, 1–8.

[11] William Severa, Craig M Vineyard, Ryan Dellana, Stephen J Verzi, and James BAimone. 2019. Training deep neural networks for binary communication withthe Whetstone method. Nature Machine Intelligence (2019), 1.

[12] Sameet Sreenivasan and Ila Fiete. 2011. Grid cells generate an analog error-correcting code for singularly precise neural computation. Nature neuroscience14, 10 (2011), 1330.

[13] Federico Stella, Peter Baracskay, Joseph O�Neill, and Jozsef Csicsvari. 2019. Hip-pocampal Reactivation of Random Trajectories Resembling Brownian Di�usion.Neuron (2019).

Documents

Composing neural algorithms with Fugu