lecture4 pipelining - Yeditepe Üniversitesi Bilgisayar...

CSE CSE 533533: : Advanced Advanced Computer ArchitectureComputer Architecturess

PipeliningPipeliningInstructor: Gürhan Küçük

Yeditepe University

Lecture notes based on notes by Mark D. Hill and John P. ShenUpdated by Mikko Lipasti

PipeliningPipelining

Forecast– Big Picture– Datapath– Control– Data Hazards

StallsForwarding

– Control Hazards– Exceptions

MotivationMotivation

Single cycle implementation– CPI = 1– Cycle = imem + RFrd + ALU + dmem +

RFwr + muxes + control– E.g. 500+250+500+500+250+0+0 = 2000ps– Time/program = N x 2ns

Instructions CyclesProgram Instruction

TimeCycle

(code size N)

(CPI) (cycle time)

Idea!Idea!We use 500 MHz clock with 2ns cycle timeBut, we can reduce cycle time and divide the task into several stages so we can speed things up– i.e. 0.5ns cycle time => 2 GHz clock

MulticycleMulticycle

Multicycle implementation:

i+4Fi+3

MXDFi+2XDFi+1

WMXDFi

987654321Cycle:Instr:

ButBut……

Multicycle implementation– CPI = 3, 4, 5– Cycle = max(memory, RF, ALU, mux, control)– =max(500,250,500) = 500ps– Time/prog = N x 4 x 500 = N x 2000ps = N x 2ns

Would like:– CPI = 1 + overhead from hazards (later)– Cycle = 500ps + overhead– In practice, ~3x improvement

Big PictureBig Picture

Instruction latency = 5 cyclesInstruction throughput = 1/5 instr/cycleCPI = 5 cycles per instructionInstead– Pipelining: process instructions like a lunch

buffet– ALL microprocessors use it

E.g. Pentium-IV, Athlon, Power4

Big PictureBig Picture

Instruction Latency = 5 cycles (same)Instruction throughput = 1 instr/cycleCPI = 1 cycle per instructionCPI = cycle between instruction completion = 1

Ideal PipeliningIdeal Pipelining

Bandwidth increases linearly with pipeline depthLatency increases by latch delays

GateDelay

Comb. Logicn Gate Delay

GateDelayL Gate

DelayL

L GateDelayL Gate

DelayL

L BW = ~(1/n)

n--2n--2

n--3n--3

BW = ~(2/n)

BW = ~(3/n)

Ideal PipeliningIdeal Pipelining

WMXDFi+4WMXDFi+3

WMXDFi+2WMXDFi+1

WMXDFi

Pipelining IdealismsPipelining Idealisms

Uniform subcomputations– Can pipeline into stages with equal delay

Identical computations– Can fill pipeline with identical work

Independent computations– No relationships between work units

Are these practical?– No, but can get close enough to get significant

speedup

ComplicationsComplications

Datapath– Five (or more) instructions in flight

Control– Must correspond to multiple instructions

Instructions may have – data and control flow dependences (coming up!)– i.e. units of work are not independent

One may have to stall and wait for another

An ExampleAn ExampleConsider an unpipelined machine. Assume that it has 20 ns clock and that it uses

cycles for execution frequencyALU 2 30%Branch 5 20%Memory 7 50%

Suppose that due to clock skew and setup, pipelining the machine adds 2 ns overhead to the clock. Ignoring any latency impact, how much speedup in the instruction execution rate will we gain from a pipeline?

DatapathDatapath (Fig. 6.11)(Fig. 6.11)

MIPS MIPS Datapath Datapath Without PipeliningWithout Pipelining

What Is What Is PipeliningPipelining

Pipelined Pipelined DatapathDatapathStart with single-cycle datapathPipelined execution– Assume each instruction has its own datapath– But each instruction uses a different part in every

cycle– Multiplex all on to one datapath– Latches separate cycles (like multicycle)

Ignore hazards for now– Data– Control Instruction�

memory

Address

Add Add�result

Shift�left 2

IF/ID EX/MEM MEM/WB

M�u�x

0Write�data

M�u�x

1Registers

Read�data 1

Read�data 2

Read�register 1

Read�register 2

16Sign�

extend

Write�register

Write�data

Read�data

ALU�result

M�u�x

ALUZero

Data�memory

Address

Pipelined Pipelined DatapathDatapath

Instruction�memory

Address

Add Add�result

Shift�left 2

IF/ID EX/MEM MEM/WB

M�u�x

0Write�data

M�u�x

1Registers

Read�data 1

Read�data 2

Read�register 1

Read�register 2

16Sign�

extend

Write�register

Write�data

Read�data

ALU�result

M�u�x

ALUZero

Data�memory

Address

The Basic Pipeline For MIPSThe Basic Pipeline For MIPS

Reg ALU DMemIfetch Reg

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 6 Cycle 7Cycle 5

Instr.

MIPS FunctionsMIPS FunctionsWhat Is What Is PipeliningPipelining

Instruction Fetch (IF):Send out the PC and fetch the instruction from memory into the instruction register (IR); increment the PC by 4 to address the next sequential instruction.IR holds the instruction that will be used in the next stage.NPC holds the value of the next PC.

Passed To Next StageIR Mem[PC]NPC PC + 4

Address

Add Add�result

Shift�left 2

IF/ID EX/MEM MEM/WB

M�u�x

0Write�data

M�u�x

1Registers

Read�data 1

Read�data 2

Read�register 1

Read�register 2

16Sign�

extend

Write�register

Write�data

Read�data

ALU�result

M�u�x

ALUZero

Data�memory

Address

Add Add�result

Shift�left 2

IF/ID EX/MEM MEM/WB

M�u�x

0Write�data

M�u�x

1Registers

Read�data 1

Read�data 2

Read�register 1

Read�register 2

16Sign�

extend

Write�register

Write�data

Read�data

ALU�result

M�u�x

ALUZero

Data�memory

Address

Instruction Decode/Register Fetch Cycle (ID):Decode the instruction and access the register file to read the registers.The outputs of the general purpose registers are read into two temporary registers (A & B) for use in later clock cycles.We extend the sign of the lower 16 bits of the Instruction Register.

Passed To Next StageA Regs[IR6..10]

B Regs[IR11..15]

Imm ((IR16) ##IR16..31

Passed To Next StageIf Memory Reference:ALUOutput A + Imm

If Reg-Reg ALU Inst.:ALUOutput A func. B

If Reg-Imm ALU Inst.:ALUOutput A func. Imm

If Branch:ALUOutput NPC + Imm<<2Cond (A op 0)

Execute Address Calculation (EX):We perform an operation (for an ALU) or an address calculation (if it’s a load or a Branch).If an ALU, actually do the operation. If an address calculation, figure out how to obtain the address and stash away the location of that address for the next cycle.

Address

Add Add�result

Shift�left 2

IF/ID EX/MEM MEM/WB

M�u�x

0Write�data

M�u�x

1Registers

Read�data 1

Read�data 2

Read�register 1

Read�register 2

16Sign�

extend

Write�register

Write�data

Read�data

ALU�result

M�u�x

ALUZero

Data�memory

Address

Passed To Next StageIf LOAD Inst:LMD Mem[ALUOutput]

If STORE Inst:Mem[ALUOutput] B

If Branch Inst:if (Cond)

PC ALUOutputelse

PC NPCMEMORY ACCESS (MEM):If this is an ALU, do nothing.If a load or store, then access memory.If this is a branch, then update PC

Address

Add Add�result

Shift�left 2

IF/ID EX/MEM MEM/WB

M�u�x

0Write�data

M�u�x

1Registers

Read�data 1

Read�data 2

Read�register 1

Read�register 2

16Sign�

extend

Write�register

Write�data

Read�data

ALU�result

M�u�x

ALUZero

Data�memory

Address

Passed To Next StageIf Reg-Reg ALU:Regs[IR16..20] ALUOutput

If Reg-Imm ALU:Regs[IR11..15] ALUOutput

If LOAD Inst:Regs[IR11..15] LMD

WRITE BACK (WB):Update the registers from either the ALU or from the data loaded.

Address

Add Add�result

Shift�left 2

IF/ID EX/MEM MEM/WB

M�u�x

0Write�data

M�u�x

1Registers

Read�data 1

Read�data 2

Read�register 1

Read�register 2

16Sign�

extend

Write�register

Write�data

Read�data

ALU�result

M�u�x

ALUZero

Data�memory

Address

Things to NoticeThings to Notice

For some instructions no processing is done in certain stages:– Register-to-register instructions: No process in

MEM stage– STORE instructions: No process in WB stage

Things to Notice Things to Notice -- IIII

Instructions may read their operands only when they are in ID stageReason: ???

Things to Notice Things to Notice -- IIII

Instructions may read their operands only when they are in ID stageReason: Reduces read port requirements on the Register File– Complexity reduction– Power/Energy reduction

Things to Notice Things to Notice -- IIIIII

Instructions may write their results only when they are in WB stageReason: ???

Things to Notice Things to Notice -- IIIIII

Instructions may write their results only when they are in WB stageReason: Reduces write port requirements on the Register File– Complexity reduction– Power/Energy reduction

Pipeline HurdlesPipeline Hurdles

Limits to pipelining: Hazards prevent next instruction from executing during its designated clock cycle– Structural hazards: HW cannot support this

combination of instructions (single person to fold and put clothes away)

– Data hazards: Instruction depends on result of prior instruction still in the pipeline (missing sock)

– Control hazards: Pipelining of branches & other instructions that change the PC

– Common solution is to stall the pipeline until the hazard is resolved, inserting one or more “bubbles”in the pipeline

Pipeline HurdlesPipeline HurdlesDefinition : Conditions that lead to incorrect behavior if not fixed

Structural hazard – two different instructions use same h/w in same cycle

Data hazard – two different instructions use same storage – must appear as if the instructions execute in correct order

Control hazard– one instruction affects which instruction is next

ResolutionPipeline interlock logic detects hazards and fixes them simple solution: stall -increases CPI, decreases performance better solution: partial stall -some instruction stall, others proceed better to stall early than late

Structural HazardsStructural HazardsWhen two or more different instructions want to use same hardware resource in same cycle

e.g., MEM uses the same memory port as IF as shown in this slide.

Instr.

Time (clock cycles)

Instr 1

Instr 2

Instr 3

Instr 4

Structural HazardsStructural HazardsThis is another way of looking at the effect of a stall.I

Time (clock cycles)

Instr 1

Instr 2

Instr 3

Bubble Bubble Bubble BubbleBubble

Structural HazardsStructural Hazards

This is another way to represent the stall we saw on the last few pages.

Structural HazardsStructural HazardsDealing with Structural Hazards

Stall – low cost, simple – Increases CPI – use for rare case since stalling has performance effect

Pipeline hardware resource – useful for multi-cycle resources – good performance – sometimes complex e.g., RAM

Replicate resource – good performance – increases cost (+ maybe interconnect delay) – useful for cheap or divisible resources

Structural HazardsStructural HazardsStructural hazards are reduced with these rules:

Each instruction uses a resource at most once Always use the resource in the same pipeline stage Use the resource for one cycle only

Many RISC ISA’a designed with this in mind Sometimes very complex to do this. For example, memory of necessity is used in

the IF and MEM stages.

Some common Structural Hazards:Memory - we’ve already mentioned this one.Floating point - Since many floating point instructions require many cycles, it’s easy for them to interfere with each other. Starting up more of one type of instruction than there are resources. For instance, the PA-8600 can support two ALU + two load/store instructions per cycle - that’s how much hardware it has available.

Structural HazardsStructural Hazards

We want to compare the performance of two machines. Which machine is faster?Machine A: Dual ported memory - so there are no memory stallsMachine B: Single ported memory, but its pipelined implementation has a 1.05 times faster clock rate

Assume:Ideal CPI = 1 for bothLoads are 40% of instructions executed

This is the example on Page 144.

Data HazardsData Hazards

These occur when at any time, there are instructions active thatneed to access the same data (memory or register) locations.

Where there’s real trouble is when we have:

instruction Ainstruction B

and B manipulates (reads or writes) data before A does. This violates the order of the instructions, since the architecture implies that A completes entirely before B is executed.

Data HazardsData HazardsRead After Write (RAW)

InstrJ tries to read operand before InstrI writes it

Caused by a “Dependence” (in compiler nomenclature). This hazard results from an actual need for communication.

Execution Order is:InstrIInstrJ

I: add r1,r2,r3J: sub r4,r1,r3

Data HazardsData HazardsWrite After Read (WAR)

InstrJ tries to write operand before InstrI reads i– Gets wrong operand

– Called an “anti-dependence” by compiler writers.This results from reuse of the name “r1”.

Can’t happen in MIPS 5 stage pipeline because:– All instructions take 5 stages, and– Reads are always in stage 2, and – Writes are always in stage 5

I: sub r4,r1,r3 J: add r1,r2,r3K: mul r6,r1,r7

Data HazardsData HazardsWrite After Write (WAW)

InstrJ tries to write operand before InstrI writes it– Leaves wrong result ( InstrI not InstrJ )

Called an “output dependence” by compiler writersThis also results from the reuse of name “r1”.

Can’t happen in MIPS 5 stage pipeline because: – All instructions take 5 stages, and – Writes are always in stage 5

Will see WAR and WAW in later more complicated pipes

I: sub r1,r4,r3 J: add r1,r2,r3K: mul r6,r1,r7

Data Flow GraphsData Flow GraphsI1 mul $15, $10, 4I2 addu $24, $6, $15I3 lw $25, 0($24)I4 mul $13, $24, $14I5 addu $14, $6, $13I6 lw $15, 0($14)

Data HazardsData HazardsSimple Solution to RAW

• Hardware detects RAW and stalls • Assumes register written then read each cycle

+ low cost to implement, simple -- reduces IPC

• Try to minimize stalls

Minimizing RAW stalls

• Bypass/forward/short-circuit (We will use the word “forward”)• Use data before it is in the register

+ reduces/avoids stalls -- complex

• Crucial for common RAW hazards

The use of the result of the ADD instruction in the next three instructions causes a hazard, since the register is not written until after those instructions read it.

Instr.

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

Time (clock cycles)

IF ID/RF EX MEM WB

Figure 3.9

Data HazardsData HazardsForwarding To Avoid

Data Hazard

Forwarding is the concept of making data available to the input of the ALU for subsequent instructions, even though the generating instruction hasn’t gotten to WB in order to write the memory or registers.

Figure 3.10

Time (clock cycles)Inst

add r1,r2,r3

sub r4,r1,r3

and r6,r1,r7

or r8,r1,r9

xor r10,r1,r11

There are some instances where hazards occur, even with forwarding.

The data isn’t loaded until after the MEM stage.

Time (clock cycles)

Instr.

lw r1, 0(r2)

sub r4,r1,r6

and r6,r1,r7

or r8,r1,r9

Figure 3.12

There are some instances where hazards occur, even with forwarding.

The stall is necessary as shown here.

Figure 3.13

Time (clock cycles)

or r8,r1,r9

Instr.

lw r1, 0(r2)

sub r4,r1,r6

and r6,r1,r7

RegIfetch ALU DMem RegBubble

Ifetch ALU DMem RegBubble Reg

Ifetch

ALU DMemBubble Reg

Data HazardsData HazardsThis is another representation

of the stall.

WBMEMEXIDIFOR R8, R1, R9

WBMEMEXIDIFAND R6, R1, R7

WBMEMEXIDIFSUB R4, R1, R5

WBMEMEXIDIFLW R1, 0(R2)

WBMEMEXIDIFstallOR R8, R1, R9

WBMEMEXIDstallAND R6, R1, R7

WBMEMEXstallIFSUB R4, R1, R5

WBMEMIDIFLW R1, 0(R2)

Data HazardsData HazardsInstruction scheduled by compiler - move instruction in order to reduce stall.

lw Rb, b -- code sequence for a = b+c before scheduling lw Rc, c Add Ra, Rb, Rc -- stall sw a, Ra lw Re, e -- code sequence for d = e+f before scheduling lw Rf, f sub Rd, Re, Rf -- stall sw d, Rd

Arrangement of code after scheduling.

lw Rb, b lw Rc, c lw Re, e Add Ra, Rb, Rclw Rf, f sw a, Ra sub Rd, Re, Rfsw d, Rd

Pipeline Scheduling

Data HazardsData Hazards Pipeline Scheduling

% loads stalling pipeline

0% 20% 40% 60% 80%

scheduled unscheduled

CControlontrol HazardsHazards

Control hazards can cause a greater performance loss for pipelines than do data hazards

A control hazard occurs when we need to find the destination of a branch, and can’t fetch any new instructions until we know that destination

Example (Example (quicksortquicksort/MIPS)/MIPS)# for (; (j < high) && (array[j] < array[low]) ; ++j );# $10 = j# $9 = high# $6 = array# $8 = low

bge done, $10, $9mul $15, $10, 4addu $24, $6, $15lw $25, 0($24)mul $13, $8, 4addu $14, $6, $13lw $15, 0($14)bge done, $25, $15

cont:addu $10, $10, 1. . .

done:addu $11, $11, -1

Control Hazard on BranchesControl Hazard on BranchesThree Stage StallThree Stage Stall

Control Hazards

10: beq r1,r3,36

14: and r2,r3,r5

18: or r6,r1,r7

22: add r8,r1,r9

36: xor r10,r1,r11

Branch Stall ImpactBranch Stall ImpactIf CPI = 1, 30% branch, Stall 3 cycles => new CPI = 1.9!

(Whoa! How did we get that 1.9???)Two part solution to this dramatic increase:

– Determine branch taken or not sooner, AND– Compute taken branch address earlier

MIPS branch tests if register = 0 or ^ 0

MIPS Solution:– Move Zero test to ID/RF stage– Adder to calculate new PC in ID/RF stage

must be fast can't afford to subtract compares with 0 are simple Greater-than, Less-than test sign-bit, but not-equal must OR all bits more general compares need ALU

– 1 clock cycle penalty for branch versus 3

In the next chapter, we’ll look at ways to avoid the branch all together.

Control Hazards

Five Branch Hazard Five Branch Hazard AlternativesAlternatives

#1: Stall until branch direction is clear

#2: Predict Branch Not Taken– Execute successor instructions in sequence– “Squash” instructions in pipeline if branch actually taken– Advantage of late pipeline state update– 47% MIPS branches not taken on average– PC+4 already calculated, so use it to get next instruction

#3: Predict Branch Taken– 53% MIPS branches taken on average– But haven’t calculated branch target address in MIPS

MIPS still incurs 1 cycle branch penaltyOther machines: branch target known before outcome

Control Hazards

Control DependenceControl DependenceOne instruction affects which executes next– sw $4, 0($5)– bne $2, $3, loop– sub $6, $7, $8

WMXDFbneWMXDFsub

WMXDFsw

Control Dependence Control Dependence -- StallStallDetect dependence and stall– sw $4, 0($5)– bne $2, $3, loop– sub $6, $7, $8

WMXDFbneWMXDFsub

WMXDFsw

2-cycle bubble

Predict not takenPredict not takenDetect dependence and stall– sw $4, 0($5)– bne $2, $3, loop– sub $6, $7, $8

WMXDFbneWMXDFsub

WMXDFsw

If loop is not taken

WMXDFbneWMXDF--mul

WMXDFsw

If loop is taken

SUB is flushed from the pipeline

WMXDFbneWMXDFmul

WMXDFsw

If loop is taken

2-cycle bubble

#4: Execute Both Paths

#5: Delayed Branch– Define branch to take place AFTER a following instruction

branch instructionsequential successor1sequential successor2........sequential successorn

branch target if taken

– 1 slot delay allows proper decision and branch target address in 5 stage pipeline

– MIPS uses this

Branch delay of length n

Control Hazards Five Branch Hazard Alternatives

Delayed BranchDelayed Branch

Where to get instructions to fill branch delay slot?– Before branch instruction– From the target address: only valuable when branch taken– From fall through: only valuable when branch not taken– Cancelling branches allow more slots to be filled

Compiler effectiveness for single branch delay slot:– Fills about 60% of branch delay slots– About 80% of instructions executed in branch delay slots useful in

computation– About 50% (60% x 80%) of slots usefully filled

Delayed Branch downside: 7-8 stage pipelines, multiple instructions issued per clock (superscalar)

Control Hazards

Evaluating Branch Evaluating Branch AlternativesAlternatives

Scheduling Branch CPI speedup v. Speedup v.scheme penalty unpipelined stall

Stall pipeline 3 1.42 3.5 1.0Predict taken 1 1.14 4.4 1.26Predict not taken 1 1.09 4.5 1.29Delayed branch 0.5 1.07 4.6 1.31

Conditional & Unconditional = 14%, 65% change PC

Control Hazards

Pipeline speedup = Pipeline depth1 +Branch frequency ×Branch penalty

Pipelining Introduction Pipelining Introduction SummarySummary

Just overlap tasks, and easy if tasks are independentSpeed Up Š Pipeline Depth; if ideal CPI is 1, then:

Hazards limit performance on computers:– Structural: need more HW resources– Data (RAW,WAR,WAW): need forwarding, compiler scheduling– Control: delayed branch, prediction

Speedup =Pipeline Depth

1 + Pipeline stall CPIX

Clock Cycle Unpipelined

Clock Cycle Pipelined

Control Hazards

Compiler Compiler ““StaticStatic””Prediction ofPrediction ofTaken/Untaken BranchesTaken/Untaken Branches

esso gc

jsp2 or

esso gc

jsp2 or

Always taken Taken backwardsNot Taken Forwards

Control HazardsThe compiler can program what it thinks the branch direction will be. Here are the results when it does so.

Compiler Compiler ““StaticStatic”” Prediction Prediction ofofTaken/Untaken BranchesTaken/Untaken Branches

Improves strategy for placing instructions in delay slot

Two strategies– Backward branch predict taken, forward branch not taken– Profile-based prediction: record branch behavior, predict branch based on

prior run

Control Hazards

Evaluating Static Branch Evaluating Static Branch Prediction StrategiesPrediction Strategies

Misprediction ignores frequency of branch“Instructions between mispredicted branches” is a better metric

100000

esso gc

jsp2 or

Profile-based Direction-based

Control Hazards

Resolution of Pipeline HazardsResolution of Pipeline Hazards

Pipeline hazards– Potential violations of program dependences– Must ensure program dependences are not violated

Hazard resolution– Static: compiler/programmer guarantees correctness– Dynamic: hardware performs checks at runtime

Pipeline interlock– Hardware mechanism for dynamic hazard resolution– Must detect and enforce dependences at runtime

Pipelined Pipelined DatapathDatapath -- SummarySummary

Instruction flow– add and load– Write of registers– Pass register specifiers

Any info needed by a later stage gets passed down the pipeline– e.g. store value through EX

Pipelined Pipelined ControlControl

Control– Set by 5 different instructions– Divide and conquer: carry IR down the pipe

MIPS ISA requires the appearance of sequential execution– True of most general purpose ISAs

Pipelined ControlPipelined Control

IF and ID– None

EX– ALUop, ALUsrc, RegDst

MEM– Branch, MemRead, MemWrite

WB– MemtoReg, RegWrite

Address

Instruction�[20– 16]

MemtoReg

Branch

RegDst

ALUSrc

16 32Instruction�[15– 0]

0Registers

Write�register

Write�data

Read�data 1

Read�data 2

Read�register 1

Read�register 2

Sign�extend

M�u�x

1Write�data

Read�data M�

ALU�control

RegWrite

MemRead

IF/ID ID/EX EX/MEM MEM/WB

MemWrite

Address

Data�memory

Add Add�result

Shift�left 2

ALU�result

ALUZero

M�u�x

Figure 6.29Figure 6.29

Control

IF/ID ID/EX EX/MEM MEM/WB

Instruction

Figure 6.30Figure 6.30

Branch

RegDst

ALUSrc

16 32Instruction�[15– 0]

M�u�x

Add Add�result

RegistersWrite�register

Write�data

Read�data 1

Read�data 2

Read�register 1

Read�register 2

Sign�extend

M�u�x

ALU�result

Write�data

Read�data

M�u�x

ALU�control

Shift�left 2

MemRead

Control

WBIF/ID

EX/MEM

MEM/WB

M�u�x

AddressData�

memory

Address

Pipelined ControlPipelined Control

Controlled by different instructionsDecode instructions and pass the signals down the pipeControl sequencing is embedded in the pipeline

PipeliningPipelining

Not too complex yet– Data hazards– Control hazards– Exceptions

Data HazardsData HazardsMust first detect hazards

ID/EX.WriteRegister = IF/ID.ReadRegister1

ID/EX.WriteRegister = IF/ID.ReadRegister2

EX/MEM.WriteRegister = IF/ID.ReadRegister1

EX/MEM.WriteRegister = IF/ID.ReadRegister2

MEM/WB.WriteRegister = IF/ID.ReadRegister1

MEM/WB.WriteRegister = IF/ID.ReadRegister2

Not all hazards because– WriteRegister not used (e.g. sw)– ReadRegister not used (e.g. addi, jump)– Do something only if necessary

Hazard Detection Unit– Several 5-bit (or 6-bit) comparators

Response? Stall pipeline– Instructions in IF and ID stay– IF/ID pipeline latch not updated– Send ‘nop’ down pipeline (called a bubble)– PCWrite, IF/IDWrite, and nop mux

dest src1 src2opcode

Detecting dependencies over memory locations– In a typical datapath, actual writes to the main

memory are serialized via write buffers

Write Buffers (FIFO)

Address Data to be written

Data fromCache

Writes to the Main Memory

Detecting dependencies over memory locations– When the effective address of a LOAD is

computed, it is compared with the effective addresses of all the writes in the Write Buffers

Write Buffers (FIFO)

Address Data to be written

Data fromCache

Writes to the Main Memory

The Effective Address of the LOAD

Wider Comparators(typically 32 bit)

A better response – forwarding– Also called bypassing

Comparators ensure register is read after it is writtenInstead of stalling until write occurs– Use mux to select forwarded value rather than

register value– Control mux with hazard detection logic

Write before Read RFWrite before Read RF

Register file design– 2-phase clocks common– Write RF on first phase– Read RF on second phase

Hence, same cycle:– Write $1– Read $1

No bypass needed– If read before write or DFF-based, need bypass

Control Flow HazardsControl Flow Hazards

Control flow instructions– branches, jumps, jals, returns– Can’t fetch until branch outcome known– Too late for next IF

What to do?– Always stall– Easy to implement– Performs poorly– 1/6th instructions are branches, each branch

takes 3 cycles– CPI = 1 + 3 x 1/6 = 1.5 (lower bound)

Predict branch not takenSend sequential instructions down pipelineKill instructions later if incorrectMust stop memory accesses and RF writes– Including loads (why?)

Late flush of instructions on misprediction– Complex– Global signal (wire delay)

Even better but more complex– Predict taken– Predict both (eager execution)– Predict one or the other dynamically

Adapt to program branch patternsLots of chip real estate these days

– Pentium III, IV, Alpha 21264

Current research topic

Another option: delayed branches (later in branch handling techniques!)– Always execute following instruction– “delay slot”– Put useful instruction there, otherwise ‘nop’

Losing popularity– Just a stopgap (one cycle, one instruction)– Superscalar processors (later)

Delay slot just gets in the way (special case)

Exceptions and PipeliningExceptions and Pipelining

add $1, $2, $3 overflowsA surprise branch– Earlier instructions flow to completion– Kill later instructions– Save PC in EPC, set PC to EX handler, etc.

ExceptionsExceptions

Even worse: in one cycle– I/O interrupt– User trap to OS (EX)– Illegal instruction (ID)– Arithmetic overflow– Hardware error– Etc.

Interrupt priorities must be supported

ReviewReview

Big PictureDatapathControl– Data hazards

StallsForwarding or bypassing

– Control flow hazardsBranch prediction

Exceptions

lecture4 pipelining - Yeditepe Üniversitesi Bilgisayar...

Documents

Lecture4 Compression

CI Lecture4

Lecture4: 123.312

Lecture4: 123.702

Lecture4 ancientart

ÖZGEÇMİŞ - Yeditepe

YEDİTEPE UNIVERSITY

202 lecture4

Dynamic Scheduling II - Yeditepe Üniversitesi Bilgisayar ...cse.yeditepe.edu.tr/~gkucuk/courses/cse533.fall04/Lecture12-Precise_Interrupts.pdfCOMPSCI 220 / ECE 252 Lecture Notes Dynamic

T.C. YEDİTEPE ÜNİVERSİTESİdocs.neu.edu.tr/library/nadir_eserler_el_yazmalari/... · T.C. YEDİTEPE ÜNİVERSİTESİ SOSYAL BİLİMLER ENSTİTÜSÜ İLKÖĞRETİM OKULLARINDA

16612 lecture4

Lecture 4kabanets/405/Slides/Lecture4.pdf · 2019. 1. 10. · Lecture 4 Thursday, January 10, 2019 12:19 PM Lecture4 Page 1 . Lecture4 Page 2 . Lecture4 Page 3 . Lecture4 Page 4

Lecture4 (1)

Yeditepe University

YEDİTEPE ÜNİVERSİTESİ HUKUK FAKÜLTESİ DERGİSİ · 2017. 9. 21. · Prof. Dr. Erhan Adal, Yeditepe Üniversitesi Hukuk Fakültesi Prof. Dr. Halûk Ülman, Yeditepe Üniversitesi,

Acemoglu lecture4

CURRICULUM VITAE - Yeditepe Mathematics

Esn yeditepe tanıtım

Lecture4: 123.101

Lecture4 blood