Embed Size (px)
DESCRIPTION
Ad multos annos Joos Vandewalle Frameless Wave Computing. Tamás Roska András Horváth and Miklós Koller Pázmány P. Catholic University, Budapest. Outline. Cellular Wave Computing Frameless spatial-temporal computing Activation controlled frameless computing - PowerPoint PPT Presentation
Citation preview
Ad multos annosJoos Vandewalle
Frameless Wave Computing
Tamás Roska
András Horváth and Miklós Koller
Pázmány P. Catholic University, Budapest
Outline
• Cellular Wave Computing
• Frameless spatial-temporal computing
• Activation controlled frameless computing
• Delayed template frameless computing
• Outlook
Cellular Wave Computing
Spatial-temporal waves combined:
Input wave
Self wave
Activation wave (e.g. stroboscopic effect)
boundary wave
in a CNN Universal Machine with non-standard CNN dynamics
The computational model
We have three wave dynamics evolving together:
• the dynamics of the spatial-temporal input flow (u)
• the self-dynamics of the computing cellular array (x defined by F)
• the dynamics of the active light-sources (v defined by G1 G2)
We are interested in their interaction in two cases:
‘independent activation’ case ‘adaptive activation’ case
))(,(or const.
)),(,(
21
1
vfvGvv
uxfxFx
))(),(,(
)),(,(
122
1
xfvfvGv
uxfxFx
u: two-dimensional input-flowx: two-dimensional computation-flow (inner state of the cells)v: two-dimensional flow defining the activation strength of the light-sources
Frameless spatial-temporal computing A. Solving an NP hard problem with a Cellular Wave Computer with sparse nonlocal connection
in one sigle wave
B. Detecting spatial-temporal events
For A: M. Ercsey-Ravasz, T. Roska, Z. Néda, „Cellular Neural Networks for NP-hard optimization”,EURASIP Journal on Advances in Signal Processing, Special issue: CNN Technology for Spatio-temporal Signal Processing, doi: 10.1155/2009/646975, 2009.M. Ercsey-Ravasz, Z. Toroczkai, "Optimization Hardness as Transient Chaos in an Analog Approach to Constraint Satisfaction", Nature Physics 7, 966 (2011) arxiv:1208.0526B. Molnár, Z. Toroczkai, M. Ercsey-Ravasz, "Continuous-time Neural Networks Without Local Traps for Solving Boolean Satisfiability", CNNA 2012, Torino, Italy (2012) doi:10.1109/CNNA.2012.6331411
Jedlik Laboratories, Pazmany University, Budapest
Problem statement for an NP complete problem
Solution of the K-SAT problem
The KSAT problem is NP complete and (widely used in the field of optimization)
For our prototype problem we have 10 state variables (xi) and 35 constraints (Ci) each of them containing three state varaibles.
A constraint can be writen in the following form:
The problem is solved if each of the constarints are satisfied in the formula.
Jedlik Laboratories, Pazmany University, Budapest
Problem statement
The example problem we have investigated can be written in the following form:
Jedlik Laboratories, Pazmany University, Budapest
The Dynamics
This heterogenous CNN network contains two type of cells (one for the state and one for the constraints) with state variables s(t) and a(t)
Where:
Jedlik Laboratories, Pazmany University, Budapest
The architecture of the network
Jedlik Laboratories, Pazmany University, Budapest
Transient behaviourThe only fixed point of this system is the solution of the logical formula
The system converges to the solution from every initial state
Jedlik Laboratories, Pazmany University, Budapest
Transient behaviourThe only fixed point of this system is the solution of the logical formula
The system converges to the solution from every initial state
Jedlik Laboratories, Pazmany University, Budapest
Transient behaviourThe state transition of all 10 state variables 1.5 means true and -1.5 means false
Jedlik Laboratories, Pazmany University, Budapest
B. Spatial-temporal event detection• No frames in biology – multichannel
visual „computing” – starting in the retina
• Dynamic spatial-temporal motifs
• Examples:looming, horizontal and vertical speed „calculated already in the retina, like an optical flow
• Combining a few wave channels
• Registration of three modalities in superior colliculus (vision, audio, touch)
Jedlik Laboratories, Pazmany University, Budapest
Activation controlled frameless computing
Use an unstable spatial-temporal self wave
Use a constant activation dynamics
Apply the reflected wave as an input
The output dynamics becomes stable in time and codes the terrain property
Jedlik Laboratories, Pazmany University, Budapest
The general scope:
the aim: to detect spatio-temporal features or events
the computational environment:a Cellular Wave Computer architecture, where the computations are done by locally propagating waves. The active light of the sensors can be adaptively tuned in spatial-temporal rule.
system setup:computational method: software simulationhardware framework: infrared lighting and sensor array
spatia-temporal algorithms
measurement and simulation results
Jedlik Laboratories, Pazmany University, Budapest
Sensor array:to collect the input-data from the scene• A) 8x8 active LED array with receiver photo sensors• B) control- and readout-circuits
Simulator:to process the raw measurement data in the afore mentioned computational model• state-equations: both explicit Euler and RK-45 methods to approximate•software framework: c++, MATLAB
System setup
Jedlik Laboratories, Pazmany University, Budapest
The particular example:
The task: to detect a specific terrain feature (a bump or a valley) which has bigger size than the sensorarray itself.
The key step: to apply the whole image flow on the input, instead of the separately captured frames (frameless detection).
Jedlik Laboratories, Pazmany University, Budapest
The computational model
We have three wave dynamics evolving together:• the dynamics of the spatial-temporal input flow (u)• the self-dynamics of the computing cellular array (x defined by F)
• the dynamics of the active light-sources (v defined by G1 G2)
We are interested in their interaction in two cases:
‘independent activation’ case ‘adaptive activation’ case
))(,(or const.
)),(,(
21
1
vfvGvv
uxfxFx
))(),(,(
)),(,(
122
1
xfvfvGv
uxfxFx
u: two-dimensional input-flowx: two-dimensional computation-flow (inner state of the cells)v: two-dimensional flow defining the activation strength of the light-sources
Jedlik Laboratories, Pazmany University, Budapest
Template-program of the computing array
An asymmetric template with few non-zero element:
0.0,0.1
,6.0,0.1,1.1:
000
00
000
00
000
zb
rpsTzzbB
r
spsA
• boundary condition: zero-flux• size of the computational array: 8 x 8 cells• computational model:
Chua-Yang
Please consider the qualitative effect of the vertical coupling (from I. Petrás; size: 41 x 23; FSR-model):
Jedlik Laboratories, Pazmany University, Budapest
Jedlik Laboratories, Pazmany University, Budapest
Delayed template frameless computing
Motivation
•Delays-time constant differences in single synapses
•Drastic delay differences between electrical and chemical synapses
•Delay differences between channels
Jedlik Laboratories, Pazmany University, Budapest
Detection of different spatial frequencies
The CNN Universal Machine architecture is capable of detecting structures (spatial
characteristic) by simple templates (operations) in a simple and elegant way
Grayscale input image Binary output image representing the different structures
Jedlik Laboratories, Pazmany University, Budapest
Detection of spatial frequencies in practice
Periodic Pattern Formation and Its Applications in Cellular Neural Networks Taisuke Nishio, Yoshifumi Nishio
Jedlik Laboratories, Pazmany University, Budapest
Frameless detection
Nyquist-Shanon sampling theorem:
If a function x(t) contains no frequencies higher than B hertz, it is completely determined by giving its ordinates at a series of points spaced 1/(2B) seconds apart.
The detection of a spatial-temporal event can be easier in the continuous time-domain if the criteria above are not fulfilled.
Temporal detection: almost always frame based
temporal changes are the differences between the frames, not the real dynamics.
Jedlik Laboratories, Pazmany University, Budapest
Frameless detection
It is difficult to identify the highest frequency in some dynamics: Tsunami
If the event is fast the (sampling and processing) detection has to be two times faster.
Jedlik Laboratories, Pazmany University, Budapest
Spatial-temporal detection in the retina
Continuous analogue processing in the retina
Our retina (brain) handles dynamics, not image sequences:Low frame-rate movies, animations
Jedlik Laboratories, Pazmany University, Budapest
Example: Looming detection
-Complex task
-Computationally expensive with regular architectures
-Simply done in the retina
- Done in an analogue, continuous way
Jedlik Laboratories, Pazmany University, Budapest
Looming
Modeling the response of the ganglion cells with a CNN chip
Jedlik Laboratories, Pazmany University, Budapest
Not only the coupling strengths,
but also coupling delays are defined.
Extension of regular CNN dynamics, the delay is defined as the delay between the elements
CNN with implicit memory
B and W templates design
Delay type CNN template
Jedlik Laboratories, Pazmany University, Budapest
Diagonal movement detection
Input video Output video
Jedlik Laboratories, Pazmany University, Budapest
Diagonal movement detection
Excites the cells temporarily: the time of excitation is controlled by the template
Jedlik Laboratories, Pazmany University, Budapest
Diagonal movement detection
The excited cells remain excited (in this case black). Detects the trajectory of an object.
Jedlik Laboratories, Pazmany University, Budapest
Detection of a given trajectory
Input VideoOutput video
The aim is to identify the object moving up in the input-flow
This task can be solved by a single delayed-cnn template
Jedlik Laboratories, Pazmany University, Budapest
Detection of a given trajectory
Input Video Output video
The previous result can be extended to identify objects moving along a given trajectory with a given speed
Jedlik Laboratories, Pazmany University, Budapest
Delayed edge detection:Identification of movement speed and direction
Input VideoOutput video
The dark edge will appear where we can detect an edge on the current input flow, while the bright edge will appear where the edge was τe time ago. This can be used to detect the speed and the direction of the moving object.
Jedlik Laboratories, Pazmany University, Budapest
Outlook
• Develop a design methodology for spatial-temporal computing without frames
• Develop a special physical mplementation framework
• Towards a 3-layer vertically integrated system
• Learning from neurobiological prototypes
