Building an Artificial Brain For Less Than $10,000

Prof. Dr. Hugo de GARIS

Head of Artificial Intelligence Group,International School of Software,

Wuhan University, Wuhan, Hubei Province, CHINAprofhugodegaris@yahoo.com

http://www.iss.whu.edu.cn/degaris

What is an Artificial Brain?

Definition :

A set of 10,000 – 20,000 evolved neural net circuit modules that are interconnected according to the designs of human “BAs” (Brain Architects).

Brain Building Course :

Prof de Garis has taught a PhD level course on “Brain Building” at his American university (Utah State) over the period 2001-2006,and will continue to teach this course at his new professorial job in China.

Neural net circuit modules can be evolved conventionally on a PC but this can typically take hours to a day or more per module.

Obviously an artificial brain (A-Brain) consisting of several 10,000s of evolved neural net (NN) modules will take too long to build using only a PC to evolve them.

Hence the need for a neural net module evolution ACCELERATOR.

There are two major approaches to accelerating NN evolution –

a) The Software Approachb) The Hardware Approach

a) The Software Approach

The typical algorithm for evolving a NN is the Genetic Algorithm (GA), but a GA can be very inefficient, because its “geneticoperators” e.g. mutation, crossover are random, and hence mostof the time they produce wasted effort.

There have been new attempts lately in GAs to generate the nextgeneration more efficiently, using statistical or “machine learning”techniques,

e.g.

EDA (Estimation of Distribution Algorithms)LEM (Learnable Evolution Method), etc

b) The Hardware Approach

Perform the GA based evolution of a NN module inspecial hardware that speeds the evolution by a factorof 10s to 100s of times compared to using software in a PC.

e.g. use a Celoxica electronic board (www.celoxica.com)that contains a Xilinx FPGA with (today) 6 mega-gates.

This FPGA is programmed using a “C-like” high level language called HANDEL-C (Handel the composer, notHandle, as in handbag).

Handel-C code is “hardware compiled” directly into theFPGA on the Celoxica board.

The GA based evolution of a neural net (with a given fitnessdefinition, i.e. a performance measure) is programmed inHandel-C and executed on the Celoxica board.

The best evolved NN on the Celoxica board is downloaded into the PC’s memory.

The 10s – 100s of times speed up factor is CRITICAL.

It makes brain building PRACTICAL and CHEAP.

The Celoxica board (6 megagate FPGA) ~ $7000

A robot to be controlled by the A-Brain ~$1000

So the total cost is less than $10,000

How to Connect the NNs into a Useful A-Brain?

Answering this question is one of the major research challenges of Brain Building.

This “Brain Building strategy for less than $10,000” is aimedat creating a methodology for creating “Artificial Brains”, that anyArtificial Intelligence (AI) research lab can afford.

Hopefully, a whole community of Artificial Brain researcherscan now arise, which will share its ideas on how to “architect” A-Brains, leading to workshops, conferences, and journals.

A current alternative, is to use expensive PC clusters, costing large amounts of money, that only rich labs and companies canafford.

e.g.

a) Switzerland’s (EPFL) and IBM’s “Blue Brain Project” thataims to simulate a cortical column at the molecular level, and scale up to the full cortex over 10-15 years, using IBM’s “BlueGene” supercomputer (that consists of a large PC cluster).

b) Artificial Development (AD)’s “CCortex Project” with its1000 processor PC cluster.

The Celoxica evolved NN route offers a much cheaper alternative.

Individual NN Modules

There are various categories of NN modules that can be evolves –

a) Pattern recognition modulesb) Motion controller modulesc) Decision making modulesd) Timing modulese) Memory modules etc

The art of being a Brain Builder (or Brain Architect) is to conceive how to evolve these individual modules, and how to connect them together to build Artificial Brains (A-Brains).

A Concrete Example of the Evolution of a NN Module –

Imagine a grid of receptors whose outputs connect to a NN asexternal inputs.

If light falls on a grid pixel, a strong output signal is sent to the connecting neuron in the NN.

If no light falls on the pixel, a weak output signal is sent.

The output signal of the NN is used to evolve the fitness.

A “multi-task” evolutionary approach is typically used forthe evolution of pattern detectors.

Assume we have a pattern on the grid of pixels we want to detect.

Call this pattern P, the “positive example”.

Let Q be a pattern similar to P. Call it the “negative example”.

(There can be many positive and negative examples).

The example P is radiated onto the grid for e.g. 100 ticks of theclock, and then pattern Q for a further 100 ticks.

The target (desired) output of the output neuron of the NN is tobe high if the pattern P is shown, and low if pattern Q is shown.

A suitable fitness definition for such a “P Detecting” NN could be -

fitness = 1/(t=1to100 (Tt – At)2 + t=101to200 (Tt – At)2 )

where At is the actual output signal at tick t, and Tt is the desired or target signal at tick t. At = 0.8 (high) or 0.2 (low)

Brain Architects soon develop considerable creativity in designingmany individual NN modules.

For details, see the PowerPoint notes of Prof. Hugo de Garis’s PhD course (CS7940) on Brain Building at

http://www.iss.whu.edu.cn/degaris

The BAs (Brain Architects) evolve large numbers of NN modules(10,000-20,000 of them), and download them one by one from the Celoxica board into the PC’s memory.

The BAs then need to specify how the NN modules connect up to form the A-Brain.

To do this, special software in the PC, called “IMSI”(Inter Module Signaling Interface) is used.

This software is essentially a set of look up tables (LUTs)that tell the PC which NN module is connected to which other NN module.

These LUTs are used for a given module to find the external signals from other NN modules.

The IMSI also uses the downloaded evolved weights Wij of each module to calculate the neural signals of each module.

Suggested Tasks of A-Brains

a) Control of Autonomous Robotsb) Visual Processingc) Speech Processingd) etc

The immediate task we have chosen is to use a 4 wheeledrobot with a CCD camera, a gripper, and a 2-way radio antennato send/receive signals from the antenna at the PC.

The A-Brain is contained in the PC. It sends control signalsvia its radio antenna to the radio antenna on the robot, to control its actions.

The task of the robot is to detect with its camera eye,UXO (UneXploded Ordnance), i.e. small cluster bomblets. It approaches them, picks them up and deposits them in some central place, so that friendly troops can march through anarea cleared of UXO.

There will be many A-Brain applications.

Disadvantages of this Approach –

a) Undesired Inter-Modular Synergy

The modules are evolved individually, and thenassembled into networks of networks.

Therefore it is possible that undesirable “synergisticsignaling” effects could happen.

What to do?

If adding a new module to an existing architecturecauses undesirable side effects, then one could re-evolveanother module.

b) No Multi-Modular Evolution

Evolving 10,000-20,000 modules can be done with a team ofBAs, but Moore’s Law will make A-Brains possible in PCswith 100,000 modules.

Obviously multi-module evolution needs to be automated.This is a new research challenge that will have to be facedin the next few years.

But first, A-Brains of 10,000-20,000 modules need to be builtto show the approach is feasible, and that useful A-Brainscan be architected and built, in a reasonable time.

Why 10,000 – 20,000 modules in a PC?

Experiments have shown that an ordinary PC can update thesignaling of all the simulated neurons in an A-Brain at therate of at least 25 Hz (signals per second per neuron), if thereare no more than 10,000-20,000 modules, with roughly20 neurons per module.

As the processing speed doubles, double the number of modulescan be placed in the A-Brain.

Even an A-Brain of 10,000 NN modules can be quitesophisticated, with hundreds of behaviors, and hundreds ofpattern recognizers, etc.

By the year 2015, only 9 years away, i.e. 6 Moore’s Law doublings, would allow an A-Brain of over half a millionmodules to be built.

This is a huge effort, and would require a large team ofBAs.

e.g. if one BA can design and evolve 1 NN module in an hourthen how many BAs would be needed to build an A-Brainof 500,000 modules in 5 years?

Under reasonable assumptions (e.g. 8 hour working days etc) The answer is about 50 people, i.e. the scale that a largeprivate company could afford.

With $100,000 salaries, that’s a salary budget of $25 million.

National Brain Projects

By the year 2021, i.e. another 4 Moore doublings, will mean about 10 million modules.

This implies about 1000 people for a 5 year project, and a salary budget of about $1 Billion.

This is the size of a national (government) project.

Prof de GARIS, intends spearheading China’s national BrainBuilding project, to build the “C-Brain”, and challenges othermajor brain building countries/regions to do the same, e.g.

A-Brain (America’s Brain), E-Brain (Europe’s Brain)J-Brain (Japan’s Brain), I-Brain (India’s Brain), etc.