1824 Memory hierarchy - apt.cs.manchester.ac.ukapt.cs.manchester.ac.uk/ftp/pub/apt/peve/PEVE05/Slides/09_MemHier.pdf · © 2005 PEVEIT Unit – ARM System Design Memory hierarchy

Memory hierarchy – v5 – 1

MANCHEstER1824

The

Uni

vers

ity

rking

© 2005 PEVEIT Unit – ARM System Design

of M

anch

este

r Memory hierarchy

❏ Outline:

❍ memory hierarchy basics

❍ on-chip RAM and caches

❍ memory management

❍ operating systems

☞ hands-on: C and assembly code interwo


MANCHEstER1824

The

Uni

vers

ity

rking


of M

anch

este

r Memory hierarchy

❏ Outline:

➜ memory hierarchy basics






MANCHEstER1824

The

Uni

vers

ity

t memory

s


of M

anch

este

r Memory hierarchy

❏ A typical system has several differensubsystems:

❍ processor registers: ~100 bytes, 1 ns

– access is a small part of a clock cycle

❍ on-chip cache or RAM: ~10 Kbytes, 5 ns

– accessed at the processor clock rate

❍ off-chip ROM and RAM: ~ Mbytes, 50 n

– access costs several processor cycles

❍ backup store: ~ Gbytes, 5 ms


MANCHEstER1824

The

Uni

vers

ity

ls of cache

faster


of M

anch

este

r Memory hierarchy

❍ There may be more or fewer (or no) leve

Registers

Memory

L2 cache

CPU

L1 cache

bigger


MANCHEstER1824

The

Uni

vers

ity

rs


of M

anch

este

r Memory hierarchy

❏ Efficient operation depends on:

❍ the right things

– the code and data

❍ being in the right place

– the on-chip memory or processor registe

❍ at the right time

– when they are in use


MANCHEstER1824

The

Uni

vers

ity

re


of M

anch

este

r Memory hierarchy

❏ Processor registers

❍ are managed directly by the compiler

❏ Cache

❍ is managed automatically by the hardwa

❏ On-chip RAM

❍ is managed by the programmer

❏ Off-chip RAM

❍ is managed by the operating system


MANCHEstER1824

The

Uni

vers

ity

…

me

ace


of M

anch

este

r Memory hierarchy

❏ The objective is to approach:

❍ the performance of the fastest memory

❍ … at the cost/bit of the slowest memory

❏ Feasible because programs display:

❍ temporal locality

– accesses to a location are clustered in ti

❍ spatial locality

– accesses are clustered in the address sp


MANCHEstER1824

The

Uni

vers

ity

rking


of M

anch

este

r Memory hierarchy

❏ Outline:


➜ on-chip RAM and caches





MANCHEstER1824

The

Uni

vers

ity

ory”) is used indded systems:

er

t


of M

anch

este

r On-chip RAM

❏ System benefits of on-chip memory:

❍ increased performance – no wait states

❍ reduced power consumption

❍ improved EMC

❏ On-chip RAM (“Tightly Coupled Mempreference to a cache in some embe

❍ it is simpler, cheaper and uses less pow

❍ its behaviour is more deterministic

❍ however it requires explicit managemen


MANCHEstER1824

The

Uni

vers

ity

hich

alues

asks for them again

ess

it is full


of M

anch

este

r Caches

❏ A cache is a small on-chip memory wautomatically:

❍ keeps copies of recently used memory v

❍ supplies these to the processor when it

– thereby avoiding an off-chip memory acc

❍ decides which values to over-write when


MANCHEstER1824

The

Uni

vers

ity

ache:

?

a cache miss?

he space be?


of M

anch

este

r Cache organization

❏ There are many ways to arrange a c

❍ separate or mixed instructions and data

❍ how much memory should be loaded on

❍ how flexible should the allocation of cac

❍ how should writes be handled?


MANCHEstER1824

The

Uni

vers

ity

ata cache

0000000016

FFFFFFFF16


of M

anch

este

r Unified instruction and d

address

registers

processor

data

copies of

instructions

data

memorycache

instructions

copies of

instructionsaddress &data

instructions &data


MANCHEstER1824

The

Uni

vers

ity

tion caches

0000000016

FFFFFFFF16


of M

anch

este

r Separate data and instruc

address

address

data

registers

processor

data

copies of

instructions

data

memorycache

instructions

copies of

cache

address instructions

address data

data


MANCHEstER1824

The

Uni

vers

ity

e

Cache line


of M

anch

este

r Direct-mapped cachAddress

compare

tag RAM data RAM

deco

der

hit


MANCHEstER1824

The

Uni

vers

ity

ache

select

Address


of M

anch

este

r 2-way set associative c

❍ two (smaller) cacheblocks

❍ two chances to storeany line

❍ better hit rate

❍ more expensive

❍ can extend to 4-way,etc.


MANCHEstER1824

The

Uni

vers

ity

he

data RAM


of M

anch

este

r Fully associative cac

❍ more places to storegiven line

❍ even better hit rate

❍ even more expensive

❍ (potentially) slower

❍ requires CAM(Content Addressable Memory)

tag CAM

hit


MANCHEstER1824

The

Uni

vers

ity

arison


of M

anch

este

r Cache architecture comp

❍ Direct mapped

– simple, cheap, fast

– subject to ‘thrashing’

– choice for large caches

❍ Set associative

– compromise

– may be 2-, 4-, 8-, etc. way

– often preferred

❍ Fully associative

– best hit rate

– slow, expensive

– choice for small caches


MANCHEstER1824

The

Uni

vers

ity

es

te is performed

memory is only


of M

anch

este

r Cache write strategi

❏ Write-through

❍ all data is written to memory;matching cache locations are updated

❏ Write-through with write buffer

❍ all data is written to memory, but the writhrough a buffer

❏ Copy-back

❍ the processor writes to the cache - mainupdated on flushes.


MANCHEstER1824

The

Uni

vers

ity

cy

on power-

4)

r

nse amplifiers

power-hungry


of M

anch

este

r Cache power-efficien

❏ What is the influence of organizationefficiency?

❍ a high hit rate minimizes off-chip activity

– hit rate increases with associativity (up to

❍ set-associative caches burn more powe

– due to the increased number of active se

❍ CAM (in fully associative caches) is also


MANCHEstER1824

The

Uni

vers

ity

cy

improved?

t associative cache

cache


of M

anch

este

r Cache power-efficien

❏ How can cache power-efficiency be

❍ use serial tag and data accesses in a se

– enable only the relevant data RAM

❍ segment the CAM in a fully associative

❍ exploit sequential address sequences


MANCHEstER1824

The

Uni

vers

ity

rking


of M

anch

este

r Memory hierarchy

❏ Outline:



➜ memory management




MANCHEstER1824

The

Uni

vers

ity

nt

n address space

ms


of M

anch

este

r Memory manageme

❏ Allows each program to run in its ow

❍ using address translation

– greatly simplifying programming

❏ protects programs from other progra

❍ using memory protection

– improving system reliability

❏ supports the memory hierarchy

❍ between the off-chip RAM and the disc


MANCHEstER1824

The

Uni

vers

ity

ss …

s


of M

anch

este

r Address translation

❏ Translates

❍ the processor’s logical (or ‘virtual’) addre

❍ … into the physical memory address

❏ There are two main schemes:

❍ segmented memory management

– variable size (usually large) segments

❍ paged memory management

– fixed size pages, usually around 4 Kbyte


MANCHEstER1824

The

Uni

vers

ity

nagement

ault

ess


of M

anch

este

r Segmented memory ma

base limit

segment descriptor table

+ >?

access f

segment selector logical addr

physical address


MANCHEstER1824

The

Uni

vers

ity

ment

concatenate

0

pageframe

data


of M

anch

este

r Paged memory manage

❏ Three stage look-up

logical address31 2122 1112

pagedirectory

pagetable

table base


MANCHEstER1824

The

Uni

vers

ity

ep all relevant

slation data to

would be too slow!

lations is kept on-chip


of M

anch

este

r Address translation

❏ Segmentation schemes generally keregisters on chip

❏ Paging schemes have too much trankeep on chip

❍ three memory accesses per translation

❍ therefore a cache of recently-used trans

– a translation look-aside buffer (TLB)


MANCHEstER1824

The

Uni

vers

ity

uffer0

0


of M

anch

este

r Translation look-aside b

logical address31 1112

logical pagenumber

hit

physical pagenumber

physical address31 1112


MANCHEstER1824

The

Uni

vers

ity

hes

or

tables change (e.g. on

ame physical address


of M

anch

este

r Virtual and physical cac

❏ A cache may use virtual (pre-MMU) physical addresses

❍ physical caches

– have fewer coherency problems

– require a translation on every access

❍ virtual caches

– must be flushed whenever the translationa process switch)

– do not support synonyms

– when two virtual addresses map to the s


MANCHEstER1824

The

Uni

vers

ity

rking


of M

anch

este

r Memory hierarchy

❏ Outline:




➜ operating systems



MANCHEstER1824

The

Uni

vers

ity

o run multiple

others (to a greater


of M

anch

este

r Operating systems

❏ Scheduling

❍ an operating system allows a machine tprograms concurrently

– sometimes owned by different users

❏ Protection

❍ each program is protected from errors inor lesser extent)

❏ Resource allocation

❍ limited resources, conflicting demands


MANCHEstER1824

The

Uni

vers

ity

MU

rogrammers!

ks


of M

anch

este

r Operating systems

❏ Hardware support is needed:

❍ memory management hardware

❍ a protected operating system mode

– to prevent unauthorized access to the M

❏ Single-user systems

❍ do not need protection from malicious p

– except, possibly, virus and network attac

❍ use protection to improve reliability


MANCHEstER1824

The

Uni

vers

ity

ating System (RTOS)

m


of M

anch

este

r Operating systems

❏ Embedded systems

❍ usually run a fixed set of programs

– these can run in a single memory space

– MMU hardware is often not needed

❍ sometimes use a small Real Time Operfor:

– scheduling

– hardware resource management

❍ sometimes run a single program

– no MMU; simple monitor operating syste


MANCHEstER1824

The

Uni

vers

ity

mbly

lled from a C program

sed


of M

anch

este

r Hands-on: C and assecode interworking

❏ Closer look at the APCS

❍ see how assembly programs may be ca

❍ look at various ways the stack may be u

☞ Follow the ‘Hands-on’ instructions

Documents

1824 Memory hierarchy - apt.cs.manchester.ac.ukapt.cs.manchester.ac.uk/ftp/pub/apt/peve/PEVE05/Slides/09_MemHier.pdf · © 2005 PEVEIT Unit – ARM System Design Memory hierarchy