If the Data Cannot Come to the Algorithm...

many cores with javasession four data locality

copyright 2013 Robert Burrell Donkin robertburrelldonkin.name this work is licensed under a Creative Commons Attribution 3.0 Unported License

S(p) = p - a (p-1)

● S(p) p

● a

"in practice, the problem size scales with the number of processors" John L. Gustafson

● Gustafson's Law...

●

●

●

●

●

●

○

○

●

●

●○

■

■

●○

○

●

●○

○

○

●○

●

●

●○

●○

■

■

○

● Commodity hardware

● Scales up to Terabyte and Petabyte○ smoothly by adding new nodes

● Map-Reduce platforms typically provide○ fault tolerance eg. retry

○ orchestration

○ redundant data storage

● Statistical resilience

When you want to be able to process big data tomorrow by adding cores or computers, adopt an appropriate architecture today.