Self Organisation: Inspiring Neural Network & IT Design

Self Organisation Inspiring Neural Network & IT Design

www.oliviamoran.me

Olivia Moran is a training specialist who specialises in E-Learning instructional design and is a certified Moodle expert. She has been working as a trainer and course developer for 3 years developing and delivery training courses for traditional

classroom, blended learning and E-learning.

Self Organisation: Inspiring Neural Network & IT Design was written as part of a group collaboration.

www.oliviamoran.me

About The Authors

Submitted For Cognitive Computing, Msc in Computing, University of Ulster 2006 Authors included:

Olivia Moran

Eric Nichols

Barry Feehily

Lisa Murphy

Self Organisation: Inspiring Neural Networks & IT Design

1. ABSTRACT In an attempt to build more sophisticated neural networks and other Information Technology (I.T.) products, the industry constantly turns to the world of Biology for inspiration. The most advanced computers in the World today, are of course humans. It is therefore no wonder that engineers and computer scientists invest such a large amount of their time examining theses biological machines and the way in which they operate. I.T. products are often constructed based on the same principles or concepts on which the human body is built. These concepts relate to areas such as sensory perception and processing, the motor system and social cognition etc, the list is endless. For the purpose of this report, however, only one issue will be explored in depth, self-organisation. This report aims to highlight the means by which the understanding gained, from the study of this issue, can influence and inspire the design of I.T. products and services. It will examine the role of sleep and its effects on self-organisation. Subsequently, the development of the Nervous System (N.S.) and the importance of self-organisation to the development process will be explored in depth. This document will briefly consider how connections are made between neurons and furthermore how these can be rewired. Self-organisation occurs at a number of levels, which will be highlighted. A comparison will be made between the N.S. of invertebrates and vertebrates in an attempt to determine the effect, if any, that the sizes of these systems exert on the self-organisation process. It will be illustrated how self-organisation can be computationally modeled. Finally, this document will give some thought to future work in this field of research. KEYWORDS: Cognitive Computing, Self-Organisation, Neural Networks, The Nervous System, Self-Organising Maps.

2. INTRODUCTION The term self-organisation is used to describe the process by which “Internal structures can evolve without the intervention of an external designer or the presence of some centralised form of internal control. If the capacities of the system satisfy a number of constraints, it can develop a distributed form of internal structure through a process of self-organisation” Cilliers (1998). In basic terms self-organisation is a process that involves the organisation of group behaviour to achieve global order. This process occurs through interactions among the group and not through external influences. According to Cilliers (1998) “This process is such that structure is neither a passive reflection of the outside, nor a result of active, pre-programmed internal factors, but the result of a complex interaction between the environment, the present state of the system and the history of the system”. The concept of self-organisation is easily illustrated using an example from nature. Examples include the reaction of hair dye to our hair and the type of activity that occurs as well as the growth of plants and animals and the creation of a sculpture by an artist. One of the more widely used examples is the hare and the lynx. A study was carried out and recorded by the Hudson Bay Trading Company in Canada between 1849 and 1930. This recorded and examined specific statistics relating to the populations of hares and lynxes. In this example, the lynx is the predator of the hare. It was concluded from this study that a decrease in the number of prey, would cause a corresponding decrease in the number of predators. This was due to the fact that a reduction in prey resulted in limited food sources and so there was not enough food to sustain current predator numbers. After a period of time, the numbers of prey grew because the amount of predators was low. However, this replenished food stocks for the predators and so there numbers began to grow yet again. This process begins over again and continues in a cyclical manner. This behaviour is seen emerging from the interaction between the lynxes and hares.


This self-organisation process also takes place in our bodies. One can easily forget that our bodies are some of the most complex systems around. Consequently it is only obvious that this is a good place in which to study the concept and gain a better understanding of it. This document aims to explore the concept of self-organisation in depth. Firstly the role of sleep is considered and the part that it plays in the self-organisation process. Self-organisation is extremely important in the development of the nervous system. It is also crucial to understanding how connections between neurons are made and rewired. This development process is explored at length. The occurrence of self-organisation at different levels is considered briefly focusing on the single and networked cell levels. A comparison is made of the N.S.’s of invertebrates and vertebrates and thought is given to how the size of these systems may impact on self-organisation. It is illustrated how self-organisation can be computationally modelled. Future work in this area is also addressed.

3. SLEEP AND ITS ROLE IN SELF-ORGANISATION IN THE BRAIN In the past people taught of sleep as a dynamic and dormant activity that was part of our every day lives. Nowadays people are more aware that sleep can actually affect our daily functioning along with our physical and mental health. A number of activities occur within the brain in order to prepare us for sleep. Firstly, within the brain, neurotransmitters, nerve-signalling chemicals, act on different neurons and control whether we are asleep or awake. These neurons are located in the brainstem, the part of the N.S. that connects the brain with the spinal cord. Here they produce neurotransmitters that keep different parts of the brain active when a person is awake. Other neurons located at the base of the brain, begin signalling the relevant neurons with the body gradually falling into a sleep state. If the later does not occur sleep deprivation results. Self-

deprivation is viewed as a lack of the necessary amount of sleep that your body requires for healthy functioning. The occurrence of sleep deprivation throughout modern day society is incredibly higher than that found four or five decades ago. This is partly due to our hectic lifestyles, jobs and of course electrical lighting. People are staying awake for longer lengths of time. Consequently, there has been a substantial decrease in the average amount of sleep each person gets. A person can be deprived of sleep by their own mind and body. Sleep is extremely important and is needed for regeneration of certain parts of the body in particular the brain so that it can function properly. When the body is asleep the brain goes through a process that consists of four different stages called the R.E.M. (Rapid Eye Movement) sleep cycles. R.E.M. sleep is the desirable sleep state characterised by rapid movements of the eyes. At certain points of the sleep process, the brain is active in different ways. These can be identified using an Electroencephalogram (E.E.G.) reader. In the first stages of sleep, the body starts to relax and the heart rate begins to slow. At this point people often feel as if they are falling or feel weightless. During the second stage of sleep it becomes evident that the brain is not acting in the same way i.e. emitting the same brain waves, as when the body was awake. This stage is where deep restful sleep occurs and the body reenergises itself. The body must go through a sufficient amount of R.E.M. cycles or else the body will be unable to reenergise itself. Consequently self-deprivation would result. The effects of this self-deprivation may include difficulty concentrating, being in a bad mood, reduced energy and a greater risk of being in or causing an accident, including fall-asleep crashes. Stage three and four are much deeper sleep states however, four is more intense than three. These stages are often referred to as slow-wave sleep or delta sleep. The reason why is evident particularly in stage four where the E.E.G. reader records slow waves of high amplitude, demonstrating a pattern of deep sleep and rhythmic continuity.


Research shows that sleep-deprivation has noticeable negative effects on things such as alertness and cognitive performance. According to Thomas et al (2000) this suggests a decrease in brain activity and function primarily in the thalamus, a subcortical structure involved in alertness and attention, as well as the prefrontal cortex, a region subserving alertness, attention, and higher-order cognitive processes. It is seen that after extended periods of wakefulness or reduced sleep, neurons can begin to malfunction. This change in the neurons can have a visible affect on a person’s behaviour. Organs such as muscles are able to regenerate themselves when a person is not asleep so long as they are resting. In this circumstance the cerebral cortex within the brain is not able to rest but rather remains alert in a state of ‘quiet readiness’. This suggests that while some stages of sleep are a necessity for the regeneration of neurons others are suited to creating new memories and the formation of synaptic connections. A study was carried out in an attempt to highlight the negative effects of sleep deprivation. Seventeen males over an eighty-five hour period of sleep-deprivation were examined. The subjects were observed four times every twenty-four hours. During this period they were asked to complete a series of addition and subtraction tasks. Polysomnographic examinations confirmed that the subjects were awake. After twenty-four hours it was reported that there was a significant decrease in global C.M.R.glu, and dramatic falls in absolute regional C.M.R.glu in several cortical and subcortical structures. The main changes occurred primarily in the thalamus and prefrontal and posterior parietal cortices located near the front of the brain (See Appendix 1). From these experiments it was concluded that short-term sleep deprivation produces global decreases in brain activity with larger reductions in activity in the distributed cortico-thalamic network mediating attention and higher-order cognitive processes. It was also complementary to studies demonstrating deactivation of these cortical regions during N.R.E.M. and R.E.M. sleep.

Scientists are now realising that sleep deprivation can affect the whole body not only the brain. A study that was carried out by Dr. Eve Van Cauter from the University of Chicago showed that failing to get the right amount of sleep could affect the chemical balances in the body as well. The study looked at a male after four hours sleep for a total of six nights. Results from blood tests showed strikingly similar results to those expected from a person with diabetes. The male’s ability to process blood sugar was reduced by a total of thirty percent, this in turn caused a drop in insulin levels. It was also reported that the male had specific levels of memory impairment. Scientists strive to come up with an answer to the question ‘how much sleep does an average person need in order for their brain to reenergise itself and ideally self-organise’. This is a difficult one to answer. It is different for everyone and is influenced by factors such as age and health. It is generally accepted that babies need an average of nineteen hours a day while teenagers need a total of nine. Adults function normally with approximately seven to eight hours however, certain individuals can limit themselves to five hours while others may require ten hours of sleep. It is clear that sleep is absolutely necessary and that without it our bodies would be unable to survive for long. One study involving rats demonstrates this point effectively. Rats are seen to live for two to three years but because of sleep deprivation and not going through the R.E.M. cycle, the rats in the experiment only lived for a total of five weeks. The rats developed low blood pressure, sores on their tails and paws as well as an impaired immune system. Researchers in recent years have become a lot more interested in this area of study. They know that gaining a better understanding of sleep and all that it encompasses will in turn result in a greater insight and knowledge of the role that sleep plays in the self-organisation process.


4. THE NERVOUS SYSTEM The human body is made up of trillions of cells that interact and work together to achieve certain outcomes. Numerous amounts of these cells join forces in order to create complex systems such as the N.S. This system is found in both animals and people and is crucial to their survival as it facilitates movement, therefore enabling them to respond to changes in their environment and adapt. The N.S. consists of two main parts, the first includes the spinal cord and the brain and is known collectively as the Central Nervous System (C.N.S.). The second part is the Peripheral Nervous System (P.N.S.) and it takes in all the bodies’ different nerves.

4.1 THE DEVELOPMENT OF THE NERVOUS SYSTEM Self-organisation is undertaken at different stages of the N.S. development process. According to Willshaw (2007) it is self-organisation that is responsible for “Generating nerve cells of the right type, in the right numbers, in the right places and with the right connections”. Such a procedure is highly complex and involves “cell division, cell migration, cell death and the formation and withdrawal of synapses”.

4.1.1 Cell Division In humans after the primitive cell layers are formed, the inner cells break into a layer of ectoderm and endoderm. A new layer called the mesoderm grows between these two layers which all then begin to work together to produce the notochord. The notochord is a cylindrical shaped structure that is responsible for organising the ectoderm layer. A number of steps are followed in the achievement of this task. Chemicals are released from the notochord; these stimulate the ectoderm so that it begins to divide. This division leads to the creation of the neural plate (See Appendix 2).

After a short period a crease or fold appears in this plate, it begins to grow and a neural groove appears. Folding action continues until the creases meet and fuse together. This fusion results in the neural tube that eventually develops into the nervous system. If development goes according to plan the neural tube closes completely. Failure to close could result in abnormalities such as Spina Bifida. Vesicles grow from the front end of the tube. These will eventually become a part of the C.N.S. During the entire process all the cells in the N.S. comply with a strict set of rules. These rules determine exactly where each cell will eventually end up and what purpose it will serve. Next, cell differentiation and division occurs. Mitosis, the process by which cells divide and thus multiple takes place at the inner part of the wall of the neural tube. Firstly, the cells move away from the wall to develop further and then return to undergo mitosis. This process of division results in a huge amount of new cells being formed and consequently the thickening of the neural tube wall. The vesicles also increase in size. The new cells will eventually develop into neurons or glial cells.

4.1.2 Cell Migration The next major step in the development of the N.S. is cell migration. Cell migration refers to the movement of cells away from where they first developed, to where they are needed. This process is an extremely complex one, requiring a high level of organisation. All cells must end up in the exact desired position. “In the developing brain, for example, primitive neuronal cells migrate out of the neural tube and take up residence in distinct layers, where they send projections (axons and dendrites) through the layers of developing cells to their final targets with which they form specific connections, called synapses that allow complex functions such as learning and memory” Cell Migration Consortium (2007).


4.1.3 Cell Death Cell death is a normal occurrence as well as a “Fundamental and essential process in development” Bähr (2006). It is necessary that some cells be sacrificed for the success of the entire process. There are many theories attempting to shed light on the reason behind cell destruction. Such theories are highlighted by Willshaw (2007) and include “Failure of neurons to find their targets, failure to make the correct connections, the elimination of entire structures that may act as transient scaffolds, removal of transient branches of the tree of lineage and lack of adequate innervation”. Such explanations fail to address all the issues relating to cell death. Another hypothesis known as ‘The Neurotrophic Hypothesis’ was constructed and is currently the most logical way in which to explain why and how cell destruction occurs. “Its principal tenet is that the survival of developing neurons depends on the supply of a neurotrophic factor that is synthesized in limiting amounts in their target fields” Davies (1996). If there is not enough neurotrophic factor present, the extra neurons produced will not be able to survive and will simply die.

4.2 Neurons And Their Connections Once the neurons find their desired position, connections have to be established. Such connections accommodate communication between the neurons. Each neuron is made up of an axon and dendrites that are crucial to the entire communication process. For example, when cell A wishes to communicate with cell B, the following sequence of events occur; Cell A using the axon transmits a message. Cell B is able to receive this message via the dendrites that act like a receptor antennae. The meeting point of the two cells is known as the synapse (See Appendix 3). Neurotransmission is also dependant on chemicals that act as a neurotransmitter and the use of electric signals to get the message across to the other cell. A synapse is made up of three main parts, the axon terminal, the synaptic cleft and the dendrite spine

(See Appendix 3). The main task of any synapse is the transformation of electrical impulses into chemical signals so that they can be transported. The beginning of this conversion process is sparked by what’s known as an action potential, which is in essence a nerve impulse. “The end part of an axon splits into a fine arborisation. Each branch of it terminates in a small end bulb almost touching the dendrites of neighbouring neurons” Zurada (1992). The nerve impulse travels down to the bottom of the axon where it stimulates the synaptic vesicles resulting in the release of neurotransmitter. This flows into the synaptic cleft filling it up. The chemical then makes its way towards the dendrites of the cell it is trying to communicate with. As a result of this, parts of the membrane open up. Through these openings ions can flow in and out. This flow of ions results in a change in voltage that is known as a postsynaptic potential. This potential can be excitatory or inhibitory. If an excitatory potential is created in the case of depolarising currents, this usually leads to the production of a second action potential. However, inhibitory potentials as with hyperpolarising currents, inhibits any further action potential. It is important to note that sometimes impulses will not necessarily travel to another neuron. “The synapses thus help regulate and route the constant flow of nerve impulses throughout the N.S.” The World Book Encyclopedia (1991).

5. SELF-ORGANISATION AT DIFFERENT LEVELS Self-organisation occurs at both a networked cell and a single cell level. The networked cell level is concerned with the construction of maps that detail the connections that exist between the nerve cells. On the other hand, at the single cell level, focus is on the elimination of superinnervation from developing muscle.


5.1 NETWORKED CELL LEVEL: SELF-ORGANISING MAPS Self-organising maps are a good example of how self-organisation occurs at a networked level. Such maps are highly ordered and consist of multiple amounts of nerve cells. This conclusion is according to Willshaw (2007), a consequence of both electrophysiological and anatomical experiments. The electrophysiological experiments are concerned with the identification of a receptive field, “An area in which stimulation leads to response of a particular sensory neuron” Levine & Shefner (1991). On the other hand the anatomical expirements focus on the “Mapping between two points in different structures … using axonal tracers. Tracers placed at one point in one structure typically label a small, circumscribed area in the target, the spatial layout of points of administration being reflected in the layout of points to which the tracers go in the target” Willshaw (2007).

5.1.1 Neural Maps Seiffert & Lakhmi (2001) define neural maps as maps which “Project data from some possibly high-dimensional input space on to a position in some output space”. These neural maps are made up of neuronal groups all of which are connected. “Two functionally different neural maps connected by re-entry form a classification couple. Each map independently receives signals from other brain maps or from the world. Functions and activities in one map are connected and correlated with those in another map. For example an input could be vision and the other from touch” Clancey et al (1994). In basic terms neural maps are simply a projection of one two dimensional area onto another.

5.1.2 Topographic Maps The neural network is capable of being trained through unsupervised learning. In such circumstances the neural network can produce maps that still retain their topological features (See Appendix 5). These maps find their inspiration from humans and

resemble those maps found namely in the human’s vertebrate visual system. “Topographic maps vary considerably from one person to another”. They serve their purpose in that “Projections from one area of the brain to another often preserve neighbour relationships so that an area smoothly and continuously maps the area which project to it” Bamford et al (2006). Topographic maps have two main distinguishable characteristics. Take for example, the organism Xenopus that is made up of recoding positions. These according to Willshaw (2007) “Can be distinguished, all arranged in topographic order. The other important attribute of such maps is that they always have a specific orientation. All retinotectal maps are arranged so that temporal retina projects to rostral tectum and dorsal retina to medial tectum”. According to Bamford et al (2006) these maps can “Form in the absence of any electrical (spiking) activity and mechanisms proposed include varied repulsion to chemicals with graded expression across the target areas. However maps can be refined in the presence of electrical activity (the spread of connection fields reduced). It can be demonstrated that a combination of spatially correlated input, recurrent connections between target neurons and Hebbian learning can produce ordered projections”. From examination of these maps it has been concluded that “Connections cannot be made by means of a simple set of instructions specifying which cell connects to which other cell, more likely, the populations of cells self-organise their connections so as to ensure the correct overall pattern” Willshaw (2007). A better understanding of these connections that form under a process of self-organisation will undoubtedly lead to the creation of more biologically plausible neural networks.

5.2 SINGLE CELL LEVEL – ELIMINATION OF SUPERINNERVATION FROM DEVELOPING MUSCLE


Many models exist which put forward different arguments that claim to offer a plausible reason for the elimination of superinnervation during the development of muscles. The most widely accepted of these is the ‘Dual Constraint Model’ Bennett & Robinson (1989). This model which “Combines competition for a pre-synaptic resource with competition for a post-synaptic resource, has been shown to be superior to others with only one type of competition” Rasmussen & Willshaw (1993). When muscle fibres are being developed, they are “Superinnervated and this pattern is transformed into one of single innervation after a few weeks” Willshaw (2007). The length of time needed to complete this process as well as the total amount of elimination that takes place, differs depending on the section of the N.S. where it occurs. This theory basically operates on the tenant that motor neurons have a particular capacity “For maintaining the structure and activity of its terminals, which is shared out among them” Willshaw (1981). Each terminal has a survival strength, however this is dynamic and is constantly regulated and fine-tuned. Those terminals with a high level of strength are given precedent over the feeble terminals. Consequently, the survival strength of the endplate of the stronger terminals is increased to the detriment of the weaker ones. Such a theory would lead one to conclude that the muscle fibres and the construction of their connections as well as the pattern that they follow, are the result of a process of self-organisation under highly competitive conditions. The patterns are therefore not formed by instructions specified in the genome.

6. THE NERVOUS SYSTEM OF VERTEBRATES AND INVERTEBRATES

Invertebrates are animals sharing one common characteristic and that is they do not have a backbone or spine. On the other hand vertebrates are all those other animals who possess this spinal column structure as part of their anatomy as in the case of fish, birds, reptiles and of course humans. Freeman (2005) points out that the “Architectures of the C.N.S. of intelligent invertebrate animals differ markedly from those in vertebrate animals”. The N.S. of the invertebrates are fairly simple in construction in contrast to the vertebrates. Take for example, an invertebrate such as the bee or the octopus that has “Parallel chains of neurons resembling a ladder located ventral to the digestive system, from which and to which the axons of motor and sensory nerves extend” Freeman (2005). They may have some type of eyes and mouth in which case are “Serviced by large collections of neurons forming the dorsal cerebrum. Axons form bi-directional connections with the ventral nerve cords around the gut, so that the esophagus runs through the brain. Perhaps this is why all higher invertebrates are restricted to a liquid diet, lest they rupture their brains by swallowing solid food” Freeman (2005). The architecture of the vertebrates is such that it does not have to deal with a limitation of this nature. “The C.N.S. forms by invagination of the dorsal surface and creates the neural tube. The posterior part forms the spinal cord while the most anterior part forms the brain” Freeman (2005). Freeman (2005) also argues that despite all the differences that might exist with the architecture of the C.N.S. of both animal types they do however, “Share the ladder-like architecture of invertebrates”. Numerous studies on invertebrates have been completed concerning the role of organisation. Research evidence suggests that the Drosophila, commonly known as the fruit fly displays very “Precise and inflexible organisation” Willshaw (2007). This would lead one to conclude that self-organisation in this small and simple N.S. of the fruit fly, is in a position whereby the “Genome can afford to specify precisely all the parameters values needed which have a smaller number of neurons” Willshaw (2007). Both small and large N.S.’s seem to display the ability to self-organise.


7. Computationally Modelling Self-Organisation Kohonen created a computational model for self-organisation in 1981. Bruske & Sommer (1995) states “Kohonen’s self-organizing feature maps, besides back propagation networks, are now the most popular and successful types of artificial neural networks”. Kohonen’s model, maps every input node to every output node (See Appendix 6, Figure 1). The first step in Kohonen’s algorithm is the initialisation the synaptic weights, which can be set to random values. The next step involves finding the winning neuron by calculating the Euclidean distance between input and output neurons. Kohonen (1997) discovered that one could find the ‘winning’ neuron by using the following formulae: Best matching node = mini {||x-mi||} =

n

j

ijj mx1

2

In this formula, x represents the input and m represents the output map. The neuron pair with the smallest Euclidean distance, the closest output node represents the winning neuron on the output map (See Appendix 6, Figure 2). The value of the closest output node is adjusted so that a smaller Euclidean distance results. The nodes in the closest output node’s neighbourhood are also updated (See Appendix 6, Figure 3). The weight of the winning neuron, as well as the weights of all the neighbouring neurons, is adjusted with the formulae (Kohonen 1997). mi(t+1) = mi(t) + hci(t)[x(t) – mi(t)] In this formula, t represents an integer representing a time interval and hci(t) represents a ‘neighbourhood function’. The process then returns to the start (See Appendix 6, Figure 1) with the updated weights. The loop continues until the network sufficiently matches the target system.

As every input node is connected to every output node, adding nodes to the system causes the network to grow exponentially. The amount of computation required to calculate large systems can quickly become too data-intensive for equations to be solved within a reasonable time-scale. As an understanding of biological systems becomes more detailed, the algorithms for describing such systems require more computation. At its most detailed, if quantum confinement can be proven to exist in the interactions within and between neurons, then quantum mechanics “Represents the ultimate tool to the modelling of bio molecular systems” Chung (2007). However, Chung writes that a quantum approach is “Formidable and is an extremely time consuming process, even with some simplifying assumptions, its applications are limited to very small systems at present”. As the speed of computational machines increases and we are “Equipped with powerful computing techniques and high-performance sensors and actuators, we want to solve much more complex (highly non-linear and high-dimensional) problems” Kecman (2001). This relates significantly to self-organising systems, as the growth of these systems is exponential. Kohonen’s (2001) argues that self-organising maps are useful for classification. On the other hand, in isolation they are not as good as other methods. Lisboa (1992) compared Kohonen’s network to other classifiers (See Appendix 7). Kohonen’s self-organising map had the worst performance of the six that were compared using handwriting digit recognition as a test case. Kohonen’s network was not the first computational self-organising map. According to Grossberg (1994), in 1976 Grossberg wrote a mathematical model of a self-organising feature map, where “Neurobiological modelling rules were articulated and restated in the familiar SOFM formalism as an algebraic winner-take-all dot product rule, and a self-normalizing synaptic weight change rule whose weights change only if they are in the neighbourhood of the winner.” Five years later, Kohonen (2001) sought to “Generalise and at the same time ultimately simplify his (Grossberg’s) system description”. With this simplification, further


work can be achieved on self-organisation using Kohonen’s network as a baseline. Examples of this can be found in both hardware and software implementations of self-organising maps. The construction of self-organising maps can be a very time-consuming process because every input must be mapped to every output node. Martinez et al (2002) have found that using systolic arrays with Kohonen’s network greatly reduces computational time, as each node can be computed in parallel on different processors. Linaker and Niklasson (2000) used a neighbourhood function of 0 (no neighbours) and an altered Kohonen network, called a Resource Allocating Vector Quantizer (RAVQ), for a robot to successfully learn its environment. The main difference between the self-organising map that Kohonen authored and the RAVQ is that the latter’s output map is dynamic. The RAVQ more closely mimics biological systems in that output nodes can be created and mapped. While Kohonen’s original network is not great for classification, as shown in Table 1, enhancements such as the RAVQ can make self-organisation more realistic and give greater performance to Kohonen’s network. Self-organising systems can be modelled using any development environment and language that has access to basic mathematical libraries. Matlab has libraries that provide functions not only for mathematics, but also specifically for self-organising maps. Three such functions include: newc – returns a new competitive layer

newsom – returns a new self-organising map

newlvq – returns a new learning vector

quantisation network for classification

These functions can be used in Matlab’s scripting language, as well as graphically using Matlab’s Neural Network Toolbox graphical user interface. A great feature of Matlab is the ability to extend its functionality with the use of Matlab *.m files. An example of this is the SOM Toolbox, a set of 141 Matlab files written by the Helsinki University of Technology. These files build upon the functions

above to greatly enhance the computational modelling of self-organising maps in Matlab. As new self-organising algorithms are created, further Matlab files can be written to continue computational modelling of biological systems with textual and graphical outputs.

8. FUTURE WORK Artificial neural networks are modelled on our perception of the way the brain processes information. As technology develops, neurologists will be able to find a more definitive understanding to self-organisation in the brain while biophysicists find a better representation of our brain at a molecular and atomic level. These findings can then be used to develop better theories and technologies for artificial neural networks. Intelligent systems’ current (third) generation models take past work on neural networks and “Raises the level of biological realism by using individual spikes” Vreeken (2002). A current active area of research in neural networks that can play a vital role in self-organisation is in dynamic synapses. The models above all use static synapses, whose values only change after the Euclidean distance has been found. With dynamic synapses, the synaptic weight can change by up to a few hundred percent dependent upon the inputs to the synapses, which has been found to be the case in biological synapses.

9. CONCLUSION This document explored in depth the issue of self-organisation. It looked at sleep and how sleep or a lack of it impacts on the body’s ability to self-organise. It examined the role of self-organisation in the development of the N.S. as well as the connections between different neurons. It considered briefly self-organisation at different levels namely the single and networked cell levels.


The N.S.’s of invertebrates and vertebrates were analysed to determine whether or not the size of both these systems have any noticeable effect on self-organisation. This document illustrated how self-organisation could be modeled computationally. Ideas for future work were also put forward. The examination of such a process can aid the construction of neural networks, especially those that aim to be self-organising or self-modifying. Such a network would be able to adapt to changes in the external environment when required. Shadbolt (2004) argues strongly that the “Insights from one subject inform the thinking in another . . . The ultimate ambition is an understanding of the C.N.S.”, advances in the field of science often result in complimentary gains in the area of computing or vice versa. There is no doubt that the computing world seeks its inspiration from the world of biology. “We see complexity all around us in the natural world – from the cytology and fine structures of cells to the organization of the nervous system . . . Biological systems cope with and glory in complexity – they seem to scale, to be robust and inherently adaptable at the system level . . . Nature might provide the most direct inspiration” Shadbolt (2004). There is no doubt that “An attempt to imitate a biological phenomenon is spawning innovative system designs in an emerging alternative computational paradigm with both specific and yet unexplored potential” Bamford et al (2006).

BIBILOGRAPHY bÄhr, M. (2006) “Brain Repair – Advances in Experimental Medicine and Biology” American Journal of Neuroradiology 27(9), pp. 2014. Bamford, S. & Murray, A. & Willshaw, D. J. (2006) “Synaptic Rewiring in Neuromorphic VLSI for Topographic Map Formation” *Internet+, Date Accessed 15 April 2007, URL: http://www.see.ed.ac.uk/~s0454958/interimreport.pdf. BENNETT, M. R. & ROBINSON, J. (1989) “Growth and Elimination of Nerve Terminals at Synaptic Sites

during Polyneuronal Innervation of Muscle Cells: Atrophic Hypothesis” Royal Society Publishing 235, pp. 299-320. Bruske, J. & Sommer, G. (1995) “Dynamic Cell Structure Learns Perfectly Topology Preserving Map” Neural Computation 7(4), pp. 845-865. CELL MIGRATION CONSORTIUM (2007) “Overview of the Migration Process” *Internet+, Date Accessed: 17 March 2007, URL: http://www.cellmigration.org/science/. Chung, S. H. (2007) “Large-Scale Dynamical Models and Estimation for Permeation in Biological Membrane Ion Channels” Proceedings of IEEE Estimation and Control of Large Scale Systems, 20(20), pp.2-23. Cilliers, P. (1998) “Complexity and Postmodernism: Understanding Complex Systems” London: Routledge. Clancey, W. J, & Smoliar, S.W. & Stefik, M. J. (1994) “Contemplating minds: A forum for artificial intelligence” London: The M.I.T. Press. DAVIES, A. M. (1996) “The Neurotrophic Hypothesis: Where Does it Stand” Biological Sciences 351(1338), pp.389-394. FREEMAN, W. J. (2005) “NDN, Volume Transmission and Self-Organization in Brain Dynamics” Journal of Integrative Neuroscience 4(4), pp. 407-421. Grossberg, S. (1994) “Letter to the editor: Physiological interpretation of the self-organizing map algorithm” *online+, Date Accessed: 22 March 2007, URL: http://www.cns.bu.edu/Profiles/ Grossberg/Gro1994KohonenLetter.pdf. Kecman, V. (2001) “Learning and Soft Computing” Cambridge: MIT Press. Kohonen, T. (1997) “Self-Organizing Maps” Berlin: Springer-Verlag. Levine, M. W. & Shefner, J. M. (1991) “Fundamentals of Sensation and Perception” 2nd ed. California: Brooks & Cole.


Linaker, F. & Niklasson, L. (2000) “Time Series Segmentation Using an Adaptive Resource Allocating Vector Quantization Network Based on Change Detection” IEEE Computer Society, Proceedings of the International Joint Conference on Neural Networks. Lisboa, P. J. G. (1992) “ Neural Networks – current applications” London: Chapman & Hall. Martinez, P. & Aguilar, P. L. & Perez, R. M. & Plaza, A. (2002) “Systolic S.O.M. Neural Network for Hyperspectral Image Classification” in Zhang, D. & Pal, S. K. (2002) “Neural Networks and Systolic Array Design” London: World Scientific Publishing Co. RASMUSSEN, C. E. & WILLSHAW, D. J. (1993) “Pre-Synaptic and Post-Synaptic Competition in Models for the Development of Neuromuscular Connections” Biological Cybernetics 68, pp. 409-419. Seiffert, U. & Lakhmi, C.J. (2001) “Self-Organizing Neural Networks: Recent Advances and Applications (Studies in Fuzziness and Soft Computing)” New York: Physica-Verlag. SHADBOLT, N. (2004) “From the Editor in Chief: Nature-Inspired Computing” IEEE Intelligent Systems 19(1), pp.2-3.

THE WORLD BOOK ENCYCLOPEDIA (1991) “The Nervous System” London: World Book Inc. Thomas, M. & Sing, H. & Belenky, G. & Holcomb, H. & Mayberg, H. & Dannals, R. & Wagner, H. & Thorne, D. & Popp, K. & Rowland, L. & Welsh, A. & Balwinski, S. & Redmond, D. (2000) “Neural Basis of Alertness and Cognitive Performance Impairments During Sleepiness - Effects of 24 Hours of Sleep Deprivation on Waking Human Regional Brain Activity” Journal of Sleep Research 9(4), pp. 335-352. Vreeken, J. (2002) “Spiking Neural Networks - An Introduction” Technical Report UU-CS-2003-008, Institute for Information and Computing Sciences, Utrecht University. WILLSHAW, D. J. (1981) “The Establishment and the Subsequent Elimination of Polyneuronal Innervation of Developing Muscle: Theoretical Considerations” Biological Sciences 212 (1187), pp. 233-252. WILLSHAW, D. J. (2007) “Self-Organisation in the Nervous System” Foresight *Internet+, Date Accessed: 17 March 2007, URL: http://www.foresight.gov.uk. ZURADA, J. M. (1992) “Introduction to Artificial Neural Systems” New York: West Publishing Company.

www.businesscompany.com

Education

Self Organisation: Inspiring Neural Network & IT Design