18 Meta Techniques in Computer Science

Kanazawa Institute of Technology

18 Meta Techniques in

Computer Science

May 1st, 2015

Jun Nakano Kanazawa Institute of Technology, Japan

http://www.nakanolab.net/

K.I.T. 18 Meta Techniques in Computer Science 1. Caching 2. Blocking vs. Non-blocking 3. Profiling 4. Pipelining 5. Speculation (Prediction) 6. Relaxation 7. Eager vs. Lazy 8. Filtering 9. Reuse

10. Parallelization 11. Coupling vs. Decoupling 12. Multilevel / Clustering 13. Lookahead / Prefetching 14. Dynamic (Adaptive) 15. Replication 16. Virtualization 17. Checkpoints / Snapshots 18. Transactions

I have been seeing some common techniques used across various disciplines of computer science. Above is the list of 18 such "meta techniques" in no particular order. In the following slides, I will describe ideas behind them and show some examples (somewhat biased toward computer architecture).

2 18 Meta Techniques in Computer Science

K.I.T. #1 Caching #2 Blocking vs. Non-blocking #3 Profiling #4 Pipelining #5 Speculation (Prediction) #6 Relaxation #7 Eager vs. Lazy #8 Filtering #9 Reuse #10 Parallelization #11 Coupling vs. Decoupling #12 Multilevel / Clustering #13 Lookahead / Prefetching #14 Dynamic (Adaptive) #15 Replication #16 Virtualization #17 Checkpoints / Snapshots #18 Transactions


K.I.T. #1 Caching Idea

Exploit » Spatial locality » Temporal locality » 80-20 rule

Examples

Data cache Instruction cache: Static & Dynamic (trace cache of Pentium 4) Buffer cache of file system and DBMS Web browser's cache Proxy server Memoization LZW compression (caching frequent patterns in a dictionary)


K.I.T. #2 Blocking vs. Non-blocking Idea

Do useful stuff while waiting for something to complete

Examples Multi-tasking (OS scheduler) Multi-threading Asynchronous I/O Asynchronous communication in MPI (Message Passing

Interface): Overlapping computation and communication Out-of-order vs. in-order execution in processor Allow cache reads while there are outstanding cache misses

(MSHR: Miss Status Holding Register)


K.I.T. #3 Profiling Idea

Use runtime information to optimize execution

Examples When optimizing code, start with "hot spots" (via gprof, etc.) Lots of work by compiler folks (e.g., code layout optimization

by Spike)


K.I.T. #4 Pipelining Idea

For better throughput

Examples Processor pipeline (fetch, decode, execute, memory, and write

back)

A

B

C

D

A

B

C

D

Task

s

Time Time Wash

Dry

Fold

Sequential Laundry Pipelined Laundry

Source: http://www.cs.berkeley.edu/~pattrsn/252S01/


K.I.T. #5 Speculation (Prediction) Idea

Make an educated guess and believe you are lucky

Examples

Branch prediction Prefetching Value prediction Speculative helper thread to hide cache miss latency of main

thread


K.I.T. #6 Relaxation Idea

What if you remove a constraint?

Examples Relaxed memory consistency in multiprocessor systems

» Sequential consistency: Intuitive for programmers but little room for hardware optimization

» Relaxed consistency: Pursue hardware optimization (e.g., write buffer) at the cost of some inconveniences to programmers

Eventual consistency for distributed hash tables


K.I.T. #1 Caching #2 Blocking vs. Non-blocking #3 Profiling #4 Pipelining #5 Speculation (Prediction) #6 Relaxation #7 Eager vs. Lazy #8 Filtering #9 Reuse #10 Parallelization #11 Coupling vs. Decoupling #12 Multilevel / Clustering #13 Lookahead / Prefetching #14 Dynamic (Adaptive) #15 Replication #16 Virtualization #17 Checkpoints / Snapshots #18 Transactions


K.I.T. #7 Eager vs. Lazy Idea

Eager » Do it now if you are sure you will have to do it later, or if it is safe

and cheap Lazy

» Otherwise, do nothing until you really have to

Examples Demand paging (lazy) Copy on write (lazy) Some data structures such as Fibonacci heap that aim at

reducing amortized cost (lazy) Just-in-time compiler (eager / lazy) Protocols in MPI (Message Passing Interface)

» Eager: Send small data eagerly to avoid negotiation cost » Rendezvous: Negotiate with receiver before sending large data


K.I.T. #8 Filtering Idea

Avoid going down to the lower level if possible

Examples Cache hierarchy

» L1 cache → L2 cache → main memory Bloom filter


K.I.T. #9 Reuse Idea

Do not repeat the same thing » If the computation of f(x) is expensive and occurs frequently,

remember the answer

Examples Memoization Dynamic Instruction Reuse (Sodani and Sohi, ISCA 1997)


K.I.T. #10 Parallelization Idea

Double the computational resources and cut the time in half » In practice, be aware of Amdahl's law

Examples

SMP Multicore Cluster of computers MapReduce


#11 Coupling vs. Decoupling Idea

Less dependency is easier to handle but choose design point wisely considering trade-offs

Examples

Cache » L1 cache: Data / Instruction (Harvard architecture) » L2 cache: Unified

Decoupling is often preferred in software engineering » Interface and Implementation (object-oriented programming) » Policy and Mechanism (computer networks) » Control plane and data plane in SDN (Software Defined

Networking) » Protocol stack of TCP/IP (physical, data link, network, transport,

and application) » Database normalization


K.I.T. #12 Multilevel / Clustering Idea

Organize things neatly

Examples Cache hierarchy Directory structure of file systems Grouping instructions in processor pipeline TCP/IP Anything that looks like a tree


K.I.T. #1 Caching #2 Blocking vs. Non-blocking #3 Profiling #4 Pipelining #5 Speculation (Prediction) #6 Relaxation #7 Eager vs. Lazy #8 Filtering #9 Reuse #10 Parallelization #11 Coupling vs. Decoupling #12 Multilevel / Clustering #13 Lookahead / Prefetching #14 Dynamic (Adaptative) #15 Replication #16 Virtualization #17 Checkpoints / Snapshots #18 Transactions


#13 Lookahead / Prefetching Idea

Right thing at the right time

Examples Hardware/software prefetching from memory Cache line


K.I.T. #14 Dynamic (Adaptive) Idea

Effective use of runtime information while running

Examples Just-in-time compilation of Java Dynamic optimization in LLVM Load balancer for Web server farm


K.I.T. #15 Replication Idea

Better reliability and/or locality

Examples Disk mirroring Replication in distributed databases Replication in distributed hash tables CDN (Content Delivery Network): Akamai, etc. GFS (Google File System)


K.I.T. #16 Virtualization Idea

Beyond physical limitations Redirection

Examples Virtual memory Virtual devices Virtual machines Virtual file systems View in database Shortcuts in Windows and symbolic links in UNIX

Virtual Illusion

Physical Entity


K.I.T. #17 Checkpoints / Snapshots Idea

Avoid losing data in case of failures / human errors » Be aware of output commit problem

Examples

Database Copy on write


K.I.T. #18 Transactions Idea

Impose atomicity for group of operations

Examples Database Transactional memory for parallel programming

» Trade-off between correctness and performance • Possible to write fast and correct programs using locks, etc.,

but error-prone • No worry about locking and unlocking in transactional

memory at the cost of some performance overhead


K.I.T. Summary 1. Caching 2. Blocking vs. Non-blocking 3. Profiling 4. Pipelining 5. Speculation (Prediction) 6. Relaxation 7. Eager vs. Lazy 8. Filtering 9. Reuse

10. Parallelization 11. Coupling vs. Decoupling 12. Multilevel / Clustering 13. Lookahead / Prefetching 14. Dynamic (Adaptive) 15. Replication 16. Virtualization 17. Checkpoints / Snapshots 18. Transactions

Some advice for you if you are to apply these meta techniques to your project: Be aware of trade-offs that exist in almost any engineering project As they say, "Devil is in the details."

It is most appropriate to consider these meta techniques as just a food for thought.


Engineering

18 Meta Techniques in Computer Science