A Heterogeneous Multiple Network-On-Chip Design: An ...A Heterogeneous Multiple Network-On-Chip Design: ... 0.5 0.7 0.9 1.1 ... Episode height = Avg. number of L1 packets injected

A Heterogeneous Multiple Network-On-Chip Design:

An Application-Aware Approach

Asit K. Mishra

Chita R. Das Onur Mutlu

Executive summary •  Problem: Current day NoC designs are agnostic to application requirements and are provisioned for the general case or worst case. Applications have widely differing demands from the network •  Our goal: To design a NoC that can satisfy the diverse dynamic performance requirements of applications •  Observation: Applications can be divided into two general classes in terms of their requirements from the network: bandwidth-sensitive and latency-sensitive - Not all applications are equally sensitive to bandwidth and latency •  Key idea: Design two NoC - Each sub-network customized for either BW or LAT sensitive applications - Propose metrics to classify applications as BW or LAT sensitive - Prioritize applications’ packets within the sub-networks based on their sensitivity •  Network design: BW optimized network has wider link width but operates at a lower frequency and LAT optimized network has narrow link width but operates at a higher frequency •  Results: Our proposal is significantly better when compared to an iso-resource monolithic network (5%/3% weighted/instruction throughput improvement and 31% energy reduction)

2

•  Channel bandwidth affects network latency, throughput and energy/power

•  Increase in channel BW leads to - Reduction in packet serialization - Increase in router power

Resource requirements of various applications - I

3

Impact of channel bandwidth on application performance


4


Simulation settings:

•  8x8 multi-hop packet based mesh network •  Each node in the network has an OoO processor (2GHz), private L1 cache and a router (2GHz) •  Shared 1MB per core shared L2 •  6VC/PC, 2 stage router


5


0 1 2 3 4 5 6 7

appl

u

wrf art

deal

sjen

g

barn

es

grm

cs

nam

d

h264

gcc

pvra

y

tont

o

libq

gobm

k

asta

r

milc

hmm

er

swim

sjbb

sap

xala

n

sphn

x

bzip

lbm

sjas

sopl

x

cact

s

omne

t

gem

s

mcf

IT (

norm

. to

64b

links

)

64b links 128b links 256b links 512b links


6


0 1 2 3 4 5 6 7

appl

u

wrf art

deal

sjen

g

barn

es

grm

cs

nam

d

h264

gcc

pvra

y

tont

o

libq

gobm

k

asta

r

milc

hmm

er

swim

sjbb

sap

xala

n

sphn

x

bzip

lbm

sjas

sopl

x

cact

s

omne

t

gem

s

mcf

IT (

norm

. to

64b

links

)


1. 18/30 (21/36 total) applications’ performance is agnostic to channel BW (8x BW inc. → less than 2x performance inc.)


7


0 1 2 3 4 5 6 7

appl

u

wrf art

deal

sjen

g

barn

es

grm

cs

nam

d

h264

gcc

pvra

y

tont

o

libq

gobm

k

asta

r

milc

hmm

er

swim

sjbb

sap

xala

n

sphn

x

bzip

lbm

sjas

sopl

x

cact

s

omne

t

gem

s

mcf

IT (

norm

. to

64b

links

)


1. 18/30 (21/36 total) applications’ performance is agnostic to channel BW (8x BW inc. → less than 2x performance inc.)

2. 12/30 (15/36 total) applications’ performance scale with increase in channel BW (8x BW inc. → at least 2x performance inc.)

8

•  Reduction in router latency (by increasing frequency) -  Reduction in packet latency -  Increase in router power consumption

Impact of network latency on application performance

Resource requirements of various applications - II


9

Simulation settings:

•  … same as last experiment

•  128b links

•  Added dummy stages (2-cycle and 4-cycle ) to each router



10

0.5

0.6

0.7

0.8

0.9

1.0

1.1

applu

wrf

art

deal

sjeng

barne

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

astar

milc

hmm

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

gems

mcf

IT (n

orm. to 2-‐cycle router)

2-‐cycle router 4-‐cycle router 6-‐cycle router



11

0.5

0.6

0.7

0.8

0.9

1.0

1.1

applu

wrf

art

deal

sjeng

barne

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

astar

milc

hmm

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

gems

mcf

IT (n



1. 18/30 (21/36 total) applications’ performance is sensitive to network latency (3x latency reduction → at least 25% performance improvement)



12

0.5

0.6

0.7

0.8

0.9

1.0

1.1

applu

wrf

art

deal

sjeng

barne

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

astar

milc

hmm

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

gems

mcf

IT (n



2. 12/30 (15/36 total) applications’ performance is marginally sensitive to network latency (3x latency increase → less than 15% performance improvement)

1. 18/30 (21/36 total) applications’ performance is sensitive to network latency (3x latency reduction → at least 25% performance improvement)


0.5

0.7

0.9

1.1

applu

wrf

art

deal

sjeng

barnes

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

k

astar

milc

hmmer

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

t

gems

mcf IT (n

orm. to 2-‐cycle

router)


Application-aware approach to designing multiple NoCs

13

0

2

4

6 ap

plu

wrf art

deal

sjen

g

barn

es

grm

cs

nam

d

h264

gcc

pvra

y

tont

o

libq

gobm

k

asta

r

milc

hmm

er

swim

sjbb

sap

xala

n

sphn

x

bzip

lbm

sjas

sopl

x

cact

s

omne

t

gem

s

mcf

IT (

norm

. to

64b

links

)


0.5

0.7

0.9

1.1

applu

wrf

art

deal

sjeng

barnes

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

k

astar

milc

hmmer

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

t

gems

mcf IT (n

orm. to 2-‐cycle

router)



14

Based on the observations: 1. Applications can be classified into distinct classes: typically LAT/BW sensitive 2. LAT sensitive applications can benefit from low network latency 3. BW sensitive applications can benefit from high network bandwidth 4. Not all applications are equally sensitive to either LAT or BW 5. Monolithic network cannot optimize both classes simultaneously

0

2

4

6 ap

plu

wrf art

deal

sjen

g

barn

es

grm

cs

nam

d

h264

gcc

pvra

y

tont

o

libq

gobm

k

asta

r

milc

hmm

er

swim

sjbb

sap

xala

n

sphn

x

bzip

lbm

sjas

sopl

x

cact

s

omne

t

gem

s

mcf

IT (

norm

. to

64b

links

)


0.5

0.7

0.9

1.1

applu

wrf

art

deal

sjeng

barnes

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

k

astar

milc

hmmer

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

t

gems

mcf IT (n

orm. to 2-‐cycle

router)



15

Solution

Two NoCs where each (sub)network is optimized for either LAT or BW sensitive applications

0

2

4

6 ap

plu

wrf art

deal

sjen

g

barn

es

grm

cs

nam

d

h264

gcc

pvra

y

tont

o

libq

gobm

k

asta

r

milc

hmm

er

swim

sjbb

sap

xala

n

sphn

x

bzip

lbm

sjas

sopl

x

cact

s

omne

t

gem

s

mcf

IT (

norm

. to

64b

links

)


Design methodology

16

Processors/L1$ Network L2$ and Mem. Controllers

Logical view of a multicore processor

Design methodology

17



1

Identify LAT/BW sensitive applications - Proposes a novel dynamic application classification scheme

1

Design methodology

18



1


1

2 Design sub-networks based on applications’ demand - This network architecture is better than a monolithic iso-resource

network

2

Design methodology

19



1


1

2 Design sub-networks based on applications’ demand - This network architecture is better than a monolithic iso-resource

network

2

DE

MU

X

Design: Dynamic classification of applications

20

Network episode Compute episode

time

Application life cycle O

utst

andi

ng n

etw

ork

pack

ets


21


time

Application life cycle

•  App. has at least one outstanding packet •  Processor is likely stalling → low IPC

Out

stan

ding

net

wor

k pa

cket

s


22


time



•  App. has no outstanding packet •  High IPC

Out

stan

ding

net

wor

k pa

cket

s


23


Epi

sode

hei

ght

Episode length time




Episode length = Number of consecutive cycles there are net. packets Episode height = Avg. number of L1 packets injected during an episode

Out

stan

ding

net

wor

k pa

cket

s


24


time




Out

stan

ding

net

wor

k pa

cket

s

Short episode ht.: Low MLP, each request is critical (LAT sensitive) Tall episode ht.: High MLP (BW sensitive) Short episode len.: Packets are very critical (LAT sensitive) Long episode len.: Latency tolerant (could be de-prioritized)

Episode length Epi

sode

hei

ght

Classification and ranking

25

Classifica(on Length Long Medium Short

Tall gems, mcf sphinx, lbm, cactus, xalan sjeng, tonto

Height Medium omnetpp, apsi ocean, sjbb, sap, bzip,

sjas, soplex, tpc

applu, perl, barnes, gromacs, namd, calculix,

gcc, povray, h264, gobmk, hmmer, astar

Short leslie art, libq, milc, swim wrf, deal

Classification: LAT/BW

Classification and ranking

26

Classification: LAT/BW

Ranking Length Long Medium Short

High Rank-‐4 Rank-‐2 Rank-‐1 Height Medium Rank-‐3 Rank-‐2 Rank-‐2

Short Rank-‐4 Rank-‐3 Rank-‐1

Ranking: Sensitivity to LAT/BW

Classifica(on Length Long Medium Short

Tall gems, mcf sphinx, lbm, cactus, xalan sjeng, tonto

Height Medium omnetpp, apsi ocean, sjbb, sap, bzip,

sjas, soplex, tpc

applu, perl, barnes, gromacs, namd, calculix,

gcc, povray, h264, gobmk, hmmer, astar

Short leslie art, libq, milc, swim wrf, deal

Network design

27

1N-128

Network design

28

1N-128 2N-64x256-ST (Steering)

Network design

29


2N-64x256-ST-RK (Steering+Ranking)

Network design

30



2N-64x256-ST-RK(FS) (Steering+Ranking and

Frequency Scaling)

Network design

31

1N-128

1N-256

2N-64x256-ST (Steering)


2N-64x256-ST-RK(FS) (Steering+Ranking and

Frequency Scaling)

1N-512 (High BW) 2N-128X128

1N-320 (iso-BW)

1N-320(FS) (iso-resource)

Analysis

32

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Performance (25 WL with 50% BW and 50% LAT)

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

33


0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

34

+18%


0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

35


+7% +18%

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

36


+5% +7% +18%

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

37


5% +5%

+7% +18%

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

38


w. 2% 5%

+5% +7% +18%

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

39


w. 2% w. 2% 5%

+5% +7% +18%

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

40


w. 2% w. 2% 5%

+5% +7% +18%

0

0.4

0.8

1.2

1.6

2

Normalize

d en

ergy

Energy (25 WL with 50% BW and 50% LAT)

- 47% -  59%

0

10

20

30

40

50

60

Wei

ghte

d sp

eedu

p

Analysis

41


w. 2% w. 2% 5%

+5% +7% +18%

0

0.4

0.8

1.2

1.6

2

Normalize

d en

ergy

Energy (25 WL with 50% BW and 50% LAT)

- 47% -  59%

Best EDP across all designs

Conclusions •  Problem: Current day NoC designs are agnostic to application requirements and are provisioned for the general case or worst case. Applications have widely differing demands from the network •  Our goal: To design a NoC that can satisfy the diverse dynamic performance requirements of applications •  Observation: Applications can be divided into two general classes in terms of their requirements from the network: bandwidth-sensitive and latency-sensitive - Not all applications are equally sensitive to bandwidth and latency •  Key idea: Design two NoC - Each sub-network customized for either BW or LAT sensitive applications - Propose metrics to classify applications as BW or LAT sensitive - Prioritize applications’ packets within the sub-networks based on their sensitivity •  Network design: BW optimized network has wider link width but operates at a lower frequency and LAT optimized network has narrow link width but operates at a higher frequency •  Results: Our proposal is significantly better when compared to an iso-resource monolithic network (5%/3% weighted/instruction throughput improvement and 31% energy reduction)

42

Thank you

43

Q? Asit Mishra

[email protected]

Backup Slides . . .

44

Other metrics considered for application classification

45

0 200 400 600 800 1000 1200 1400 1600 1800 2000

0

20

40

60

80

100

120 ap

plu

wrf art

deal

sj

eng

barn

es

grm

cs

nam

d h2

64

gcc

pvra

y to

nto

libq

gobm

k as

tar

milc

hm

mer

sw

im

sjbb

sa

p xa

lan

sphn

x bz

ip

lbm

sj

as

sopl

x ca

cts

omne

t ge

ms

mcf

Sla

ck (i

n cy

cles

)

L1/L

2 M

PK

I

L1MPKI L2MPKI Slack

Analysis of network episode length and height

46

0 2 4 6 8 10 12 14

applu

wrf

art

deal

sjeng

barnes

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

k

astar

milc

hmmer

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

t

gems

mcf

Avg. episode

height

(network packets)

0 1000 2000 3000 4000 5000 6000

applu

wrf

art

deal

sjeng

barnes

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

k

astar

milc

hmmer

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

t

gems

mcf Av

g. episode

length

(in cycles)

Short length/height Medium length/height Long length/High height

0.3M 10K

0.4M 18K

Analysis of network episode length and height

47

0 2 4 6 8 10 12 14

applu

wrf

art

deal

sjeng

barnes

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

k

astar

milc

hmmer

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

t

gems

mcf

Avg. episode

height

(network packets)

0 1000 2000 3000 4000 5000 6000

applu

wrf

art

deal

sjeng

barnes

grmcs

namd

h264

gcc

pvray

tonto

libq

gobm

k

astar

milc

hmmer

swim

sjbb

sap

xalan

sphn

x

bzip

lbm

sjas

soplx

cacts

omne

t

gems

mcf Av

g. episode

length

(in cycles)

Short length/height Medium length/height Long length/High height

Based on performance scaling sensitivity to bandwidth and frequency

0.3M 10K

0.4M 18K

Empirical results to support the classification

48

SPECjbb (sjbb) as cut-off for BW/LAT sensitive applications


49



50



51



52


Why 9 clusters?

0 25 50 75

100 125 150 175 200 225

0 5 10 15 20 25 30 35

With

in g

roup

sum

of

squa

res

Number of clusters


53


Why 9 clusters?

0 25 50 75

100 125 150 175 200 225

0 5 10 15 20 25 30 35

With

in g

roup

sum

of

squa

res

Number of clusters

13x


54


Why 9 clusters?

0 25 50 75

100 125 150 175 200 225

0 5 10 15 20 25 30 35

With

in g

roup

sum

of

squa

res

Number of clusters

Analysis with varying workload combinations

55

0.7

0.9

1.1

1.3

1.5

0% BANDWIDTH 100% LATENCY




100% BANDWIDTH 0%

LATENCY

WS

and

IT (n

orm

. to

1N-1

28 n

et.) WS IT

Comparison to prior works

56

0.8

1.0

1.2

1.4

1N-1

28-S

TC

1N-1

28-S

T+R

K

2N-1

28x1

28-L

D-B

AL

2N-6

4x25

6-W

-LD

-BA

L

2N-6

4x25

6-S

T+R

K(F

S)

WS

and

IT (n

orm

. to

1N-1

28 n

et.) WS IT

Dynamic steering of packets

57

0%

20%

40%

60%

80%

100% ap

plu art

barn

es

nam

d

gcc

libq

asta

r

hmm

er

lesl

ie

calc

ulix

sjen

g

sap

sphn

x

lbm

sopl

x

omne

t

mcf

ocea

n

% p

acke

ts in

sub

-net

wor

k Latency-optimized network Bandwidth-optimized network

Design: Putting it all together

58



MU

X


59



MU

X

Classify applications based on sensitivity to network BW/LAT


60

Episode LEN/HT



MU

X


Use network episode length/height to dynamically identify

apps


61

Episode LEN/HT

Design LAT/BW optimized networks



MU

X



apps


62


Episode LEN/HT

Design LAT/BW optimized networks



MU

X


apps Prioritization within

networks

Summary

63

•  A NoC paradigm based on top-down approach (application demand/requirement analysis)

•  An efficient design paradigm for future heterogeneous multicores

Summary

64



Small core

GPGPUs

Accelerators/ ASIC

Latency critical

Throughput (BW) critical


Latency critical (real-

time constraints)

Big core

Summary

65



Small core

GPGPUs

Accelerators/ ASIC

Latency critical

Big core

Providing all these guarantees in one network is hard

Multiple networks: each customized for one metric



Latency critical (real-

time constraints)

Summary

66


•  An efficient design paradigm for future heterogeneous multicore m/c

Latency

Throughput

Local communication

Long haul comm.

Power

1 cycle/ bufferless/Faster

routers

1 cycle/ high bandwidth

Power efficient links/DVFS router

Hybrid/ fewer connectivity

network

Butterfly/express channels

MU

X

Share 2D space or 3D layers

Episode LEN/HT/??

Documents

A Heterogeneous Multiple Network-On-Chip Design: An ...A Heterogeneous Multiple Network-On-Chip Design: ... 0.5 0.7 0.9 1.1 ... Episode height = Avg. number of L1 packets injected